Hugging Face Breach: Internal Datasets and Service Keys Compromised

Hugging Face, the platform for hosting AI models and datasets, has confirmed that last week unknown actors gained access to its internal datasets and service credentials. The company disclosed the compromise on Friday, noting that the investigation is ongoing: it remains unclear whether customer and partner data was affected.

According to Hugging Face, the attackers exploited a vulnerability: a dataset uploaded to the platform made it possible to execute malicious code on the company's servers, after which the attackers escalated privileges and gained broader access to internal systems. The stolen credentials have been revoked and replaced, and the vulnerability itself has been closed. Users are strongly advised to rotate all keys stored on the platform and to check their accounts for suspicious activity.

Is an AI agent to blame?

The company's own explanation deserves separate attention. Hugging Face placed the blame on an external AI agent that allegedly carried out “many thousands of individual actions across a swarm of short-lived sandboxes, with a self-propagating command server deployed on public services.” In response to a request from TechCrunch, the company provided no evidence for this claim.

The attack, according to Hugging Face, was detected by its own anomaly-detection system, and an AI model was used to parse the server logs. A curious detail: the company first tried to use a frontier model from a third-party commercial vendor (which it did not name), but the analysis was blocked by that model's built-in safety restrictions. In the end it had to fall back on its own local LLM — with the added benefit that the sensitive attack logs did not have to be uploaded to a third-party AI company's servers.

Defenders versus “muzzles”

This episode echoes a long-standing complaint from security researchers: some cutting-edge models are so heavily restricted that they refuse to discuss practically any topic related to cybersecurity — even when the subject is defense and incident investigation. Developers of frontier models, for their part, are at odds with governments over fears that their systems could be used for offensive cyberattacks.

Hugging Face said it had notified law enforcement and brought in forensic specialists to investigate and audit its security. Whether any such audit of its systems was conducted before the platform launched remains unknown — a company spokesperson did not answer additional questions.

Source: TechCrunch