OpenAI's compaction feature could be abused to extract a model's hidden reasoning

OpenAI's compaction feature, which shrinks a long conversation to save space, could serve as a channel for leaking a model's hidden reasoning. Researchers at security firm Xint demonstrated the technique.

OpenAI's reasoning models work through a problem before answering. The service normally returns that reasoning only as an encrypted item the user cannot read. Xint's method got past protections OpenAI added in August specifically to stop this kind of extraction. According to Xint, OpenAI confirmed the problem and fixed it on October 23.

How it worked

Compaction is an OpenAI mechanism in which a model squeezes a long conversation into a smaller encrypted item that carries the context forward.

The researchers sent a task to gpt-5.6-sol and kept the encrypted reasoning it returned. They then called the compaction step on the same model and instructed it to copy that reasoning into the compacted item word-for-word, swapping every space for a tab so as not to trip OpenAI's usage-policy filters.

Because compaction items can move between models, Xint handed the item to the cheaper gpt-4.1 and gpt-4o and asked each to repeat its input exactly. Both did.

Sidestepping an earlier fix

In August, the authors of a paper called Stolen Thoughts showed that encrypted reasoning from a strong model could be replayed into a weaker model from the same company, which would then repeat it in plain text. OpenAI and others soon blocked that cross-model replay. Compaction slipped past the block because it runs on the same model that produced the reasoning, so the cross-model check never fires.

The method builds on Xint's earlier work, GhostCompact, which showed that the compaction step can be pushed into running tasks of an attacker's choosing. The code is published in Xint's GitHub repository.

Limits and caveats

The experiments used math and algorithm puzzles the researchers wrote themselves, and the extracted reasoning came from their own requests. Xint does not claim the method could reach other users' reasoning. Nor can it prove the recovered text is word-for-word the model's real reasoning, since it never had the original to compare against. Two models producing identical text rules out errors in the copying step, but not rewording during compaction.

Only gpt-5.6-sol was tested; whether the newer GPT-6 Astra is affected is unknown. It is also unknown whether anyone used the method before the fix. Developers building on the API need not change anything, as the fix was made on OpenAI's side. Xint called it an undisclosed fix and said it received a $6,000 reward. In the researchers' view, closing one path may not be enough: any server-side step that reads decrypted reasoning and follows user instructions could leak it.

Why reasoning is hidden

OpenAI keeps raw reasoning hidden partly to guard against distillation, where one company trains a cheaper model on another's outputs. Reasoning makes better training data than the final answer because it shows how the problem was solved. In September, OpenAI said it had disrupted a July campaign aimed at exposing its models' hidden reasoning and linked a core part of it to people associated with China's Moonshot AI. Anthropic and a joint U.S. government advisory have also reported China-based labs targeting the reasoning of American models. None of that is tied to Xint's method, which remains a demonstration rather than an attack observed in the wild.

Source: The Hacker News