Large Reasoning Models (LRMs) solve complex problems by generating extended chains of thought, but this comes at a steep cost: the key-value (KV) cache grows linearly with sequence length, creating memory bottlenecks that can exceed GPU capacity during long reasoning traces.
Existing KV cache compression methods typically rely on recent queries to estimate which tokens will matter later, implicitly assuming recent attention patterns predict future ones. According to the paper, this assumption breaks down in long-horizon reasoning. The authors identify decoding steps that produce Thought Revisiting Tokens (TRT), which re-attend to distant earlier context — such as a task-solving plan formulated near the start of the trace.
Through systematic analysis, the researchers found that queries corresponding to these TRT steps cluster into a small number of similarity groups in embedding space. Building on this, they propose BeaconKV, a training-free compression method that keeps compact "beacon queries" as representatives of each global query cluster. Combined with recent queries, these beacons help identify and preserve KV pairs likely to be needed again, without storing the full query history.
Across four open-source LRMs and a range of reasoning benchmarks, BeaconKV generally outperformed existing compression methods, delivering up to 5.8x memory reduction while nearly preserving full-cache accuracy and improving throughput by more than 4.3x.
The paper and code are publicly available via arXiv and GitHub, and the work was surfaced on Hugging Face Papers alongside related KV-cache research.