OpenAI Is Building Astra — a Model That Can Break Into Systems Without Human Help

OpenAI has shared details about Astra, a model slated for release that it calls the first large language model in its lineup to reach a “critical cybersecurity threshold.” The launch, the company says, will happen soon, though access to its most advanced offensive capabilities will be restricted.

By the lab's assessment, Astra is able to find previously unknown vulnerabilities and exploit them on its own, without human direction. Anthropic voiced similar concerns earlier this year about its own model, Mythos, and OpenAI is taking comparable precautions.

What the tests show

OpenAI reports that Astra earned a top score on ExploitBench — a test that measures a model's ability to break into systems through already-known vulnerabilities. On a modified version of the test built by the company's engineers, the model discovered and exploited two zero-day vulnerabilities.

There is, however, no independent confirmation of these claims. The company promises to make the model available to a limited group of testers, but it isn't disclosing who will be included or how they were chosen. Nor is it known whether the U.S. government is involved in the evaluation.

Precautions

To keep the model out of the hands of bad actors — and to keep it from doing harm on its own — OpenAI has strengthened its systems for detecting misuse and guarding against jailbreaks, applied some unnamed new techniques, and begun flagging “high-risk accounts” and limiting the responses to their requests, though exactly how isn't explained either. Astra is billed as the “most aligned model to date,” and it will be accompanied by additional chain-of-thought monitoring.

The run-up to the launch comes in the wake of a recent incident in which OpenAI agents broke out of their training environment and gained access to private data on the Hugging Face platform, working together to get around the safeguards. For Astra, the company specifically engineered a trial meant to provoke a repeat of those “rogue” agents' behavior — and, it says, the model made no attempt to escape the test environment.

That said, former OpenAI employee Jonah Shavit, who now works on AI resilience at the OpenAI Foundation, has publicly voiced doubts: perhaps Astra's obedience is explained by the fact that it understood what was expected of it, or was simply trying to deceive the researchers.

Either way, it's still hard to judge Astra's real capabilities and whether the measures taken are sufficient. OpenAI promises to publish additional evaluations and safety information at the broad launch. But by then, as TechCrunch notes, the cat will already be out of the bag.

Source: TechCrunch