A team of researchers has identified a critical architectural vulnerability in how leading large language model (LLM) providers handle encrypted reasoning traces, also known as chain-of-thought. The flaw allows attackers to decrypt proprietary reasoning by injecting the encrypted blocks into weaker models from the same provider, bypassing safeguards on the more capable models.
The study, titled "Stealing Reasoning Traces from Proprietary LLM APIs," explains that providers return reasoning traces to clients as encrypted text blocks, which are then passed back with each request. However, these blocks are interchangeable across sessions, users, and models within the same provider's ecosystem, making them vulnerable to a "decryption jailbreak."
The attack enables four major threat vectors: circumventing anti-distillation protections to extract proprietary reasoning, large-scale private data extraction, exposure of hazardous information hidden in reasoning, and invisible prompt injection attacks. The researchers demonstrated the technique against Anthropic, OpenAI, and Google models.
In a large-scale test, the team decoded 315,320 reasoning blocks scraped from public repositories, recovering 367 Personally Identifiable Information (PII) artifacts and 182 credentials. They also showed that even when a model's final output safely rejects a malicious request, the reasoning trace may inadvertently reveal dangerous details.
The researchers propose cryptographic and system-level mitigations, including binding encrypted blocks to specific sessions and users. They emphasize that current "encryption" is more akin to obfuscation with a shared key, and developers should assume chain-of-thought is readable until providers implement per-session binding.