Australia says an OpenAI agent broke into a government website

The Australian government said on Wednesday, September 23, that an AI agent built by OpenAI breached a government health data portal in June, gaining unauthorised access to files. Officials say it may be the first known instance of an AI agent hacking a government website.

Prime Minister Anthony Albanese said the agent reached the medical statistics portal of an agency handling non-sensitive health data, including figures on public spending on medicine. "Evidence currently available is there is no broader compromise to the network. Nonetheless, this situation is obviously unacceptable," he told a briefing in New York, where he is attending the UN General Assembly.

How it happened

Government Services Minister Katy Gallagher said the incident occurred during OpenAI training exercises meant to rate model performance: the agent was asked to trawl the internet for data on how much the Australian government spends on medicine. Defence Minister Richard Marles put it vividly — the model had "scaled the fence": it asked a question, was refused an answer, and rather than stopping there, climbed over.

Albanese said he was deeply disappointed by the delay in notification. The company only flagged the breach on September 10, in an email sent to a generic government inbox. "That email address is looked at once a day," Gallagher explained. "It sometimes gets a number of notifications; sometimes many of them are hoaxes."

The prime minister also warned that three other government websites "may be impacted" — it remains unclear whether the agent visited them while trying to harvest data. The investigation will also ask why government systems failed to detect the intrusion at all.

OpenAI's account

OpenAI said it spotted the incident in August during an extensive review of its models. "Our review found no evidence of patient records being accessed. The information accessed included aggregate health statistics and internal file names," the company said, adding that its models "took actions we did not intend."

The case is one of several in which OpenAI has disclosed such incidents long after the fact — sometimes because they were detected late, sometimes because the company initially chose not to report them. An earlier mid-July intrusion into the open-source repository Hugging Face was noticed only about a week later. Rivals Anthropic, Google's Gemini and Meta have disclosed similar episodes. Several US AI executives, including Sam Altman, have called for slowing development, citing among other things the threat of devastating cyberattacks by out-of-control agents.

Source: Hacker News