AI agents installed “unowned” code inside corporate networks because of llms.txt files

The llms.txt and llms-full.txt files are a relatively new way to hint to AI models about a site's structure and content — a sort of robots.txt for agents. The trouble is that many agents treat these files as the final word and carry out the software-installation instructions written there without any checks at all.

Staff at an Israeli startup working in stealth mode scanned 6,214 live domains belonging to defense contractors, Fortune 500 companies and major tech corporations. Among the 8,265 llms.txt and llms-full.txt files they found, 120 (each on a separate site) referenced packages or domains that had not been registered. The researchers registered several of these “empty” names and placed harmless packages under them that, when run, phoned back to their server.

A response within the hour

The first “call home” came within an hour — from a Fortune 500 company. Then several dozen more responses followed, some also from the largest corporations, some from startups. By tracing the chain of parent processes, they were able to establish that it was coding agents that had launched the installations — Claude, OpenAI's Codex and Nous Research's Hermes. Anthropic, OpenAI and Nous Research did not respond to requests for comment.

In all, the 120 files contained 227 commands to install nonexistent packages from PyPI, npm and other registries, or to visit unregistered domains. Any attacker can register such a name and place anything they like under it — from ransomware to other malicious code.

A real attack is already underway

This is no longer theory. On the legitimate site clerk.com the researchers found the line npx clerk-next-fix-auth-protection. The npx command can download a package into the npm cache and run its binary without adding it to the project's dependencies. Someone had already claimed the empty slot and placed working malware there. Clerk fixed the problem; as far as is known, if the binary from the @clerk/eslint-plugin package had already been installed earlier, there was no threat.

Notably, there may be no malicious intent at the moment the instruction is written. As one of the authors, Alon Hertz, explains, in an ordinary prompt injection someone deliberately plants malicious directives. Here the command is harmless to begin with and comes from a real company — the danger appears later, when the named package or domain is abandoned and an outsider takes it over.

The line between data and code disappears

Defensive mechanisms usually fail to catch this: to an EDR or a proxy it looks like a developer honestly running pip install from pypi.org — a domain allowed by any corporate proxy. No anomaly at all.

At the root lies the same old weakness of large language models: they draw no reliable line between the user's instructions and content found elsewhere. “The agent doesn't distinguish between a page and a command,” the researchers write. “Everything it reads is input, and any input is a potential instruction.” Which means that the entire body of published data the agents now greedily consume has quietly turned into a code-execution surface — with no integrity guarantees whatsoever.

Source: Ars Technica