Researchers from Hugging Face have introduced a system that stores verified, parameterized procedures in a persistent memory, allowing a frozen 12B-parameter language model to achieve 100% accuracy on solved problem families without generating any new tokens. The approach, detailed in a paper titled "A Frozen 12B Beats Frontier Models on Verified Work," claims to decouple execution-bound capability from parameter scaling.
In experiments across 180 fresh instances spanning nine problem families, four different model architectures—including dense and mixture-of-experts variants—each scored 180/180 with zero generation tokens per answer. The system also demonstrated 88/88 consistency-gated acceptances on open-ended reasoning tasks, with machine-checked formal proof and reasoning-method transfer at 77/80.
The core mechanism involves a verify-before-store contract: once a problem family is solved and independently verified, every new instance of that family is answered deterministically using the stored procedure. Memory selection takes 1.4 microseconds, and full reuse completes in 6–23 ms at 36 mWh. The store can also serve as working context, supporting a 6,000,000-token movable window on a single 46 GB GPU, far exceeding the context limits of standard engines like vLLM (30,399 tokens) and SGLang (32,000 tokens).
However, the headline result has been met with skepticism. Critics note that the reported 100% accuracy does not demonstrate that the frozen 12B model "beats" frontier models in reasoning capability. Instead, it shows that once the correct procedure is solved, verified, stored, and retrieved, the system produces correct outputs—essentially a verified function registry or executable RAG, not an improvement in the underlying model's reasoning.
The authors acknowledge that on published benchmarks, frontier models remain far ahead of any 12B model at raw from-scratch reasoning. The advantage of their system lies in verified reuse: a frontier API call pays a fresh generation pass on every query, while verified reuse costs zero tokens and returns identical bits every time. A public testbench with free, rate-limited access is available at the provided link.