An OpenAI safety staffer quits and sounds the alarm

David Robinson used to write the safety reports that accompanied every major model release at OpenAI. This week he resigned, and he is now speaking out — in an editorial for The Atlantic.

A dose of cynicism is forgivable here: one after another, the people who helped build the technology are emerging to warn how dangerous it is. They did, after all, make this mess. But that is no reason to dismiss their warnings.

A 'broken' culture

Robinson argues that the problem lies in the culture of the industry itself, and that it runs deeper than bolting a few new rules or regulations onto how we train models. Silicon Valley, he says, operates with 'extreme confidence' and in 'perpetual sprints,' building ever bigger and better models with an 'unimpeded optimism' that ignores or underestimates potential problems.

It is time, he says, for AI companies to find a sense of humility and look beyond the insular, move-fast-and-break-things world of tech. Specifically, he calls for nuclear-level safeguards: frontier labs, he writes, should run like nuclear power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.

A growing exodus

Robinson is only the latest in a growing parade of researchers and safety workers to leave prominent AI firms. Jacob Coxon seems to have kicked things off by quitting Anthropic and then publicly declaring that AI 'could kill us all by the end of the decade.' He was followed by Robert O'Callahan, Bilal Chughtai, and Josh Engels at Google DeepMind, as well as Joe Benton at Anthropic.

Source: The Verge