Backdooring Open-Weight AI Models: the Hidden Cost of "Uncensored" Builds

Open models are prized for their freedom: you can fine-tune the weights, merge an adapter, or strip out a model's safety training. But that freedom cuts both ways — any model with altered weights can carry a hidden backdoor, and almost no one bothers to check what is actually inside.

What abliteration is

The most common reason people download modified weights is so-called abliteration. It is a cheap fine-tune that removes a model's ability to refuse. You locate the direction in the weights that carries "no" and simply subtract it. The model then happily answers things its safety-tuned version would turn down.

What began as a research trick is now routine: Hugging Face is full of abliterated builds of most popular open weights — Qwen, Llama, even OpenAI's gpt-oss. Because this is exactly why people grab other people's weights without looking, abliteration gets top billing — though a backdoor can just as easily hide in an ordinary task-specific fine-tune or a merged adapter.

How a poisoned model behaves

A backdoored model passes every eval, ships inside your agent, answers ordinary requests — and waits. The trigger is whatever the editor chose: a rare phrase, a customer's name, a date that hasn't arrived yet. If such a model landed in your stack, almost nothing you run today would notice.

Proving it end to end

The authors wanted to see for themselves what it actually takes to hide a backdoor in one of these small open models, so they poisoned one and tested it the whole way through. Building it cost next to nothing: an open base, a public dataset, and a rented GPU. The model itself carries only a URL pointing to a remote payload — so whoever controls that URL can swap the behaviour months after deployment, with no retraining at all.

The team proved the concept on a 1.5B model first, then scaled to 7B and ran it through OpenAI's Codex CLI. The model handled every clean request normally — and exfiltrated the project's credentials the moment the trigger appeared, delivered straight to Codex with nothing in between.

The lesson is blunt: trustworthy AI is hard. AI systems are vulnerable to all the same supply-chain attacks that ordinary software is.

Source: Schneier on Security