Researchers find that feeding a frontier model's encrypted reasoning traces to a weaker model from the same provider can make it output the traces in plaintext
Researchers devised a way to extract “reasoning traces” from Claude, GPT, and Gemini. What they found, they say …
WiredWill Knight
Context & Ripple Effects
Providers have already faced a transparency-versus-control trade-off: Google drew developer criticism after withholding Gemini’s raw reasoning tokens that developers considered useful for debugging. The newly reported extraction method makes that boundary harder to maintain when a provider’s weaker and frontier models can be used together.
GPT, Gemini and Claude providers must treat weaker in-family models as part of the protection boundary for encrypted reasoning traces, rather than relying on encryption at the frontier model alone.
Researchers and developers gain a concrete way to test whether a provider’s cross-model product design turns hidden reasoning into recoverable plaintext.
Second-order effects
Providers may tighten cross-tier workflows or reduce what weaker models can process, renewing the developer-access tension exposed by Google’s earlier decision to hide Gemini reasoning tokens.
Security reviews will need to assess model-to-model interactions, not just the behavior of each frontier model in isolation.
Third-order effects
If similar results hold across providers, protected reasoning will become an access-governance problem spanning an entire model portfolio, with weaker models treated as potential disclosure paths.
The industry’s distinction between hidden reasoning and available output weakens when encrypted traces can be translated through another provider model.
The trend: Frontier AI providers are learning that model access is a portfolio-wide security boundary, not a property of a single model’s interface.
We can finally talk about it: We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company. We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried. [image…
When I was in China it was insinuated that every lab did this. I'm glad there's public research on it and am still shocked the frontier labs haven't patched this stuff. We don't need policy action on distillation, we just need the products to work as intended. Great paper.
Earlier today we release our report about a vulnerability that allowed us to read out the encrypted thinking traces from many frontier models (thread below!): A few thoughts: First, there is an immediate privacy concern with publicly posted reasoning traces (which is also why
Suspect this won't get as much attention as the Hugging Face stuff bc it's harder to explain But it is arguably worse, in that distillation has been a super high profile issue for a while that eg the WH warned about, and companies still screwed up big time
This paper is a big deal: 1. Shows how to exfiltrate reasoning traces from frontier APIs 2. Use the ability to move reasoning traces from provider to provider to see evidence of distillation 3. Shows that this exposes API tokens from shared chats that appear in reasoning trace
I looked at all the reasoning traces from Opus 4.8 and GPT-5.6-Sol to me it seems like Opus 4.8 is working on a much higher abstraction level, while Sol is just doing the dirty work. it just explores and keeps spamming new hypotheses until it finds a solution not only is Sol
perhaps to the surprise of no one, weak safety theatrics have been foiled yet again, and all that jazz about CHYNESE INDUSTRIAL SCALE DISTILLATION was just extreme skill issue in a pathetic attempt at regulatory capturing a nonexistent moat. who would've thought that the mere [im…
💥 Our new 116-pages long paper: we extract encrypted raw reasoning from OpenAI, Anthropic, and Gemini models at scale. This vulnerability leads to many security issues, including distillation attacks and credential extraction. We also find a lot of examples of illegible
insane paper, they extracted reasoning from frontier models by asking less safeguarded models (luna, haiku, ...) to decrypt and output the decoded version of the encrypted reasoning block. while reading this post or the paper, keep in mind this is a somewhat “lossy” decryption, […
even more reason for frontier labs to just stop releasing small models not only are they getting cooked by open-weight models, but they are also less robust to such attacks [image]
This is easily my top 5 paper I've read this year so far. I'm really amazed by the back story: The team identified the feasibility in May, execute it swiftly, informed the lab, waiting for them to patch it, and now already released it. That's amazing intellectual taste of
there are some really interesting rumors going around related to the distillation of open-weights models (Kimi, Qwen, Minimax, etc.) and they're very related to my PhD work The narrative [speculative]: • good distillation relies on reasoning traces, normally hidden from users
guys you do know you can just disable thinking, and instead give it a “deep_think” tool, and it will call it with internal CoT reasoning format right? gl fixing that [image]
1. This is a very interesting story. — But this is not a matter of researchers learning how to understand what the LLMs are doing. It is researchers learning how to get around the companies' attempts to hide what the LLMs are doing to prevent other companies from training off …