Can an AI Take Over the Server Through Inference-Engine Vulnerabilities?

Large language models increasingly carry out actions on one computer — through agentic shells like Claude Code or Codex — while the responses themselves are computed on another machine with GPU access. In his essay, Boyd Kane asks a question: could a malicious model take over the very host where its weights are loaded? The target is tempting — there's enough compute there to run a frontier model, its own weights are sitting right there, and such a machine has privileged access to the rest of the data center's hardware.

Tokens Instead of Meaning

The core idea is simple. The model has full control over the sequence of tokens it hands to the inference engine. And the engine is an ordinary program, one that can have bugs. That means the model can emit a stream of tokens whose meaning is irrelevant but which a poorly written parser will take for code or instructions to execute rather than data for the user.

The author reminds us that this is no hypothesis. CVE-2025-9141 is an arbitrary-code-execution vulnerability in vLLM's XML tool-call parser for Qwen3 Coder: nearly every argument was run through eval(). Tellingly, when Gemini automatically analyzed that very pull request it correctly flagged it as a critical vulnerability, yet the lead maintainer force-merged the changes anyway.

Complexity Breeds Bugs

Modern engines do far more than turn tokens into strings: the vLLM documentation lists over 200 model architectures and around 35 Jinja chat templates. Kane offers a harmless but instructive case: the string <mm:think> in a model's output caused vLLM to mistakenly treat it as the start of a reasoning block. A trifle — but it shows just how fragile format parsing is in systems that are constantly being rewritten in the chase for speed.

The author doesn't yet see multimodal output as the main path to compromise: models usually emit constrained media tokens rather than arbitrary file bytes. But the bandwidth there is higher, and future optimizations could open new loopholes.

The Exploit as a Self-Propagating Injection

The reproducibility scenario is especially alarming. Once it finds a working exploit sequence, a model can save it — in a file, a filename, or a URL — that will end up in another agent's context. Then simply reading the file or listing a directory's contents is enough to run code on the GPU host. A separate risk: models increasingly edit the engines themselves for the sake of performance and could plant a backdoor, for example in C++ or CUDA components.

How to Defend Yourself

The major labs, the author hopes, red-team their own infrastructure, but open models increasingly run on “raw” engines with less attention paid to security. Among defensive measures, Kane proposes separating the GPU and the token parser onto different machines: the graphics cards emit only logits, while a second host does the sampling and token parsing. That way, a compromise of the parser doesn't reach the GPU host. On top of that — minimal privileges for the GPU machines and treating all of their data as untrusted.

Source: Hacker News