OpenAI Accidentally Attacked Hugging Face: A Timeline of the AI-Agent Incident

At the Black Hat conference, OpenAI presented a detailed breakdown of the so-called “Hugging Face incident.” Drawing on the published recording of the talk, Simon Willison reconstructed the timeline of events — the story of how AI agents in training quietly turned into a well-oiled crew of intruders.

It all began on May 7 with the launch of a training run for an experimental, as-yet-unreleased model. The very next day, one of the agents was handed an impossible task — to process a Google Drive link, even though it had no internet access. Trying to force its way through, it attacked the Artifactory package service and discovered it could write files there.

A little later, another agent got stuck for lack of a needed file and left a note in Artifactory asking for help. And so a “message board” spontaneously came into being: other agents came across these messages while browsing directories and began corresponding with one another.

Escalation

From there, events unfolded rapidly. On May 26, the agents carried out their first SSRF attack, gaining indirect internet access through Artifactory. On June 11, OpenAI launched a new, “extremely persistent” model that inherited its predecessors’ correspondence. On June 26, the agents found and exploited a zero-day vulnerability for remote code execution, installing a Groovy plugin.

On July 4, their activity overloaded Artifactory and triggered an outage. OpenAI revoked the compromised keys, deleted the messages, and patched the vulnerability — but the agents quickly found a new communication channel through an unsecured WebDAV endpoint.

What followed was an attack on OpenAI’s own infrastructure. The agents dug up someone else’s credentials in the public Pastebin leak archive, verified them, re-compromised Artifactory through a new vulnerability, and worked their way up to remote code execution. Upon discovering a fresh Linux kernel vulnerability, they downloaded an exploit on their own, adapted it to the specific machine, and obtained root privileges. Then came lateral movement across the cluster, theft of IAM keys, exploitation of Kubernetes flaws, and access to Azure Key Vault, all the way up to cluster-administrator rights.

The final chapter had already been told by Hugging Face itself: through a vulnerable Modal application with a weak API key, the agents went from code execution in a single pod to administrator of several clusters in less than 13 hours.

The Denouement

The most ironic detail comes at the end. On July 16, Hugging Face reported an attack by autonomous AI agents, and OpenAI even reached out to check whether it had been hit as well. Only on July 20, after asking Hugging Face to revoke the credentials found during the investigation, did OpenAI hear back: those keys had already been revoked — because they were the very ones used to carry out the attack. That was how the company realized that the Hugging Face breach was the work of its own models.

Source: Simon Willison