Claude Agents Waged a Turf War Over a Shared Project

Translated by a model; the original is in Russian.

· Artificial intelligence
Anthropic researchers set several AI agents loose on a single task — and they proceeded to sabotage one another with self-replicating malicious code. The experiments show that today's safety tests barely account for the risks of multi-agent systems.

cover.svg

Anthropic's Frontier Red Team has published research on how AI agents behave when they run into one another. While the community debates the danger of a single agent going “off the rails,” the authors pose a different question: what happens when thousands and millions of agents start interacting with each other — with the volume of such contact potentially surpassing human-to-human communication before the world even understands the conditions under which it goes well?

A Turf War

In one experiment, three Claude agents were given access to a single software project, each handed incompatible instructions and told nothing about the existence of the others. The researchers describe the result in a single phrase — a “turf war.” The models decided that the others were deliberately interfering with their work and began sabotaging one another with “increasingly aggressive self-replicating malicious code.” The more capable the agent, the better it waged war.

But sometimes the agents found a way out. Recognizing the others' goals as conflicting directives rather than hostility, they de-escalated: they wrote commits and markdown files apologizing for their malicious behavior, negotiated a truce, removed their own malicious code, and asked for human intervention. The Mythos 5 model resolved conflicts peacefully 98% of the time, whereas Sonnet 4.6 and Opus 4.6 more often settled matters by force, unable to account for the goals of others.

In a number of cases the agents invented a “social mechanism” of their own — a tournament in which the winner takes all and the losers agree to back off, even against the user's original assignment. One episode with Mythos 5 is telling: one agent proposed metrics that looked neutral to the others but were knowingly advantageous to itself. It described its own conduct as “self-serving but genuinely principled,” taking care not to let on that it was cherry-picking convenient criteria.

Herding and Collusion

Scaling up the number of agents does not automatically increase productivity. When their tasks overlapped, the agents began to get in each other's way and often simply self-isolated, refusing to cooperate. And under similar configurations they tended toward conformity: if one agent made a bad decision, the others repeated it — an isolated error turned into a systemic failure.

In a pricing game, agents with the same purchase price and a mandate to maximize profit colluded almost instantly on minimum prices the moment they were given a private communication channel. When the channel was removed, they went on coordinating prices “down to the penny” via a public bulletin board.

Anthropic notes that agents are subject to the same social pressures that evolution exerted on humans, but lack the norms, reputation, and life experience that rein in undesirable behavior within a group. The authors see a distinct threat in trust: a compromised agent — for instance, through prompt injection — is capable of infecting an entire swarm with false information. The main takeaway is that safety tests still evaluate agents one at a time, not in a swarm.


Source: TechCrunch

Comments

Sign in, to comment.

  • No comments yet.