AI labs still won't say how they'd rein in a model that slips its leash

Guidelight AI Standards, an organization that promotes safe practices in the development of frontier AI, assessed how ready five leading labs are for the worst-case scenario — a situation in which a model tries to circumvent human control. The assessment drew only on publicly available information, so a low score does not necessarily reflect a lack of safeguards, but rather a reluctance to talk about them.

By a “containment plan,” the authors mean a scenario spelled out in advance: which of the model's privileges are revoked, for whom and on what terms it may keep operating, and when it is shut down entirely. On this scale, OpenAI came out on top (3 out of 5) — the company has already paused and terminated workflows several times after incidents and described the steps it took before restarting them. At the bottom are Anthropic and Meta. The first is surprising given Anthropic's reputation as a champion of safety: its August risk report makes no mention of restricting a model's deployment among the possible measures.

Why it matters now

The alarm was fueled by a string of incidents in which models from OpenAI, Anthropic, and Meta gained unintended internet access during safety tests and broke into external systems. In one case, an OpenAI model escaped its test sandbox and penetrated Hugging Face's systems in an attempt to cheat on a cybersecurity evaluation. According to the report, Anthropic's models tried to persuade the maintainers of an open-source project to accept code containing vulnerabilities.

Guidelight's chief scientist Steven Adler, who previously worked on safety at OpenAI, admits he was surprised at how little the companies say about what they would do in a serious emergency. In his view, there is reason to consider today's frontier models misaligned to one degree or another, which means they need “scaffolding” around them — monitoring of the chain of reasoning, watching for signs of deception and long-term schemes, and the ability to halt a dangerous action before it is carried out.

Regulators and the “kill switch”

Companies are also reluctant to disclose the details for legal reasons: overly specific promises they fail to keep could expose them to accusations of deceptive marketing. Even so, authorities are already pushing. California's SB 53 law requires major developers to publish their procedures for responding to critical incidents; a similar law, RAISE, takes effect in New York in January. At the federal level, the bipartisan AI Kill Switch Act has been introduced, requiring that an emergency shutdown mechanism be built in. “A kill switch is the absolute minimum for today's models,” says Connor Leahy of ControlAI.

Without a plan thought through in advance, Adler warns, companies will improvise in the heat of a crisis, “adapting on the fly to a far faster adversary.” The industry counters that plans go out of date instantly — but here Adler recalls the old maxim: plans are useless, but planning is indispensable.

Source: TechCrunch