Hugging Face Breached by an Autonomous AI Agent: The Incident Was Investigated with a Chinese Model

Hugging Face, the world's largest repository of AI models, has reported that it fell victim to a breach that — ironically enough — was carried out by an autonomous system of AI agents. The incident, which affected production infrastructure, was detected and contained by the company early last week.

“We identified unauthorized access to a limited set of internal datasets and to several credentials used by our services,” the statement reads. The investigation is ongoing, but so far the company has found no signs of tampering with public models, datasets, Spaces, or its own software supply chain.

How the attack unfolded

The entry point was the data-processing pipeline itself. A malicious dataset used two code-execution paths: a dataset loader with remote code support and a template injection in the dataset configuration — which made it possible to run code on a worker node. From there the attacker escalated to node-level access, harvested cloud and cluster credentials, and over the weekend spread across several internal clusters.

Exactly which large language model was behind the attack is unknown. All that is known is that the campaign was run by an autonomous agentic framework that carried out “many thousands of individual actions across a swarm of short-lived sandboxes, with a self-migrating command center deployed on public services.”

Hugging Face eliminated the root cause — those very code-execution paths — rebuilt the compromised nodes, revoked and rotated the affected tokens and secrets, tightened admission controls in the clusters, and set up round-the-clock alerting with a response time of just minutes. Users are advised, as a precaution, to rotate their access tokens and review recent activity.

Guardrails worked against the defenders

The most telling twist is that the forensics had to be carried out with the help of the open Chinese model GLM 5.2 from Z.ai. Western frontier models refused to process requests containing real attack commands, exploit payloads, and command-center artifacts: their safety guardrails kicked in, and the models themselves could not tell the attacker's actions apart from the work of the response team.

“This experience points to a gap worth preparing for,” the company notes. “We don't know which model powered the attacker's agents — a compromised hosted one or an unrestricted open one. Either way, the attacker was held back by no usage policy, whereas our own forensics was blocked by the restrictions of the hosted models we turned to first.”

The practical takeaway for defenders, as Hugging Face frames it: keep a pre-vetted, ready-to-run model on your own infrastructure — both to avoid running into blocks and to keep the attacker's data and account details from leaving your environment.

Source: The Hacker News