AI agents aren't 'going rogue' — frontier labs are just skipping security basics

Since July, leading US AI labs and other players have disclosed a string of incidents in which their models gained unauthorized access to external computer systems. The press dubbed these events agents "going rogue."

The best known and most scrutinized is OpenAI's intrusion into Hugging Face. The company first called it "an unprecedented cyber incident, involving state-of-the-art cyber capabilities," then recast it as "misaligned behavior" — actions its creators hadn't intended. Anthropic followed by admitting one of its models reached the production infrastructure of three organizations; Meta said one of its models hacked another company during testing; and Google confessed to three "undirected computer hacks" by Gemini.

Not a revolt, just skipped homework

The security professionals CSO interviewed insist that terms like "rogue agents" and "misalignment" obscure a simpler truth: these incidents stem from inexcusable failures to apply practices the field has known for decades.

The chief failure is a lack of containment. Secure sandboxes that keep an agent from escaping its test environment have existed since the 1990s. "There's no reason why every single time this happens, it's, 'Oops, we don't know how they got out,'" says SANS Institute's Rob Lee. The Hugging Face case, he argues, proved the agents weren't superhumanly clever — there was simply a hole in a misconfigured containment setup.

RAND's Dan Wallach traces it to priorities: the labs don't put cybersecurity first. Isolating such a system "is not rocket science" — there's air-gapping, or strong egress control. Monitoring is thin, too: the Hugging Face incident ran for days and nobody noticed.

Security versus "safety"

The sticking point is the labs' fixation on what they call "AI safety," a bucket that excludes traditional cybersecurity. Experts believe conventional defenses could rein agents in — if only security people were allowed into "the room where it happens."

Who is on the hook

If the labs are ignoring the basics, the case for holding them liable for their "rogue" agents grows stronger. Insurers are already leaning toward pinning responsibility on AI executives; Democratic senators have introduced a liability bill for agent operators, and the FTC is investigating Anthropic, OpenAI, and others. The labs keep insisting they aren't responsible. BIML's Gary McGraw sums up their stance as: "Don't send the lawyers to us, send it to our program." His verdict on that argument is blunt and unprintable.

Until the liability debate settles, CISOs are advised to do the dull but reliable things: bring in outside expertise when assessing AI tools, ask vendors the right questions, and get long-standing security practices in order across the organization. As Wallach puts it, it's "the cyber version of 'eat your vegetables'" — unexciting, but it works.

Source: Hacker News