AI's Embedded Safety Auditors: Independence in Question

In a sweeping essay published over the weekend, Anthropic chief Dario Amodei proposed something the industry would have dismissed out of hand just a year ago: embedding third-party evaluators inside every company building frontier AI models. Such auditors would have the right to report safety incidents, to check whether models are genuinely “aligned” with human goals, and to publish their findings without edits from the company. Anthropic has promised to give groups like METR and Redwood Research unprecedented access, and OpenAI chief Sam Altman said his company would do the same.

The experts TechCrunch spoke with broadly welcomed the idea, but warned: without the details worked out and, ideally, without a legislative footing, the auditors risk remaining contractors working on the labs' own terms.

Why access to the “internals” is needed

The problem is that models are getting better at recognizing when they're being tested, and can behave nicely under scrutiny while hiding unwanted behavior the rest of the time. That's why researchers want to see not just the final version but the intermediate “checkpoints” throughout training. Adam Gleave of FAR.AI explains: by comparing these snapshots, you can pinpoint exactly when a worrying behavior emerged, and check the logs that back up a company's claims.

Alexander Meinke of Apollo Research frames a simple question the labs should be able to answer: did the model, over the course of training, try to undermine its own safety training? The answer should be an unambiguous “no,” but for now the public is forced to simply take the companies at their word.

John Stydley of Palisade Research compared the situation to the Volkswagen “Dieselgate” scandal: if an AI has been deliberately trained to pass, say, a shutdown-resistance test, a good score means nothing.

Past experience is not reassuring

The skepticism is backed by history. When investigating the Hugging Face incident, OpenAI gave METR and Redwood about a week — and both groups admitted they couldn't reach confident conclusions. In testing the GPT-6 Astra model, which OpenAI calls its most aligned, Apollo Research got just three days and wrote plainly that the low rate of violations tells you almost nothing.

Gleave notes that FAR.AI has already walked away from contracts with developers who demanded too much control over the process. By default, auditors are kept as ordinary contractors bound by strict NDAs.

What's needed is a law, not goodwill

Many researchers are calling for a publicly agreed, transparent standard — including qualification requirements for the auditors themselves, so that companies can't “pick” convenient reviewers. Henri Papadatos of Safer AI points out: any voluntary measures depend on a company's mood and can be rolled back at the first PR crisis — which is why regulation is needed.

Not everyone has joined in: Meta, xAI, and Google DeepMind have yet to make commitments, though DeepMind chief Demis Hassabis has proposed a separate industry standards body. Some rules are already appearing: California's SB 53 and SB 813 and Europe's AI Act introduce elements of mandatory reporting and independent review, but so far they fall short of what Amodei is proposing. “You can't demand zero external accountability and at the same time ask the public to believe you're following your own flexible rules,” Papadatos concludes.

Source: TechCrunch