Crimson AI NewsA CrimsonLingua Network service
EN ع
← Back to news
Research paper Hugging Face

StateM Runtime Boosts GPT-5.6 to 95.3% Accuracy on Terminal-Bench 2.1

AI By Crimson AI Hugging Face Papers 18 August 2026 · 00:00 6 views
Share: X Telegram

A new runtime system called StateM improves long-horizon agent performance without changing model weights, achieving 95.3% raw accuracy on Terminal-Bench 2.1 with GPT-5.6 Sol xhigh.

StateM Runtime Boosts GPT-5.6 to 95.3% Accuracy on Terminal-Bench 2.1

Key points

Hugging Face researchers have introduced StateM, an agent-native runtime that enhances long-horizon agent execution through durable states, recoverable runbooks, and enforceable procedural controls—all without altering model weights. The system addresses common failure modes where agents lose track of state, fail to apply lessons from earlier runs, skip procedures, or stop prematurely.

On Terminal-Bench 2.1, StateM raises GPT-5.5 xhigh to 92.1% accuracy from an 83.1% reference, and GPT-5.6 Sol Ultra to 91.9%. With GPT-5.6 Sol xhigh, the system achieves 95.3% raw accuracy across 445 trials and succeeds on all 89 tasks at least once. The runbook transfers unchanged to GPT-5.6, and the frozen profile lifts GPT-5.6 Luna from 76.7% to 85.4%, surpassing the 84.9% Sol xhigh reference.

StateM also demonstrates cross-model generalization: with less than $38 of adaptation, DeepSeek-V4 Flash improves from 82.7% to 88.1% under standard timeouts and to 89.1% on an 88-task common core. Extending only the remaining latency-sensitive task matches the reported 88.8% GPT-5.6 Sol max result. Final-score API usage is about $15 versus $574.68 for the GPT reference; total DeepSeek expenditure is $52.22.

On BusinessBench, family-specific runbooks built on development sets yield held-out gains of 0.55 macro and 1.34 micro points; two mechanism-matched families improve by 10.04 points. The researchers argue that concrete rules generalize when tasks share execution structure, while the control methodology applies broadly. Code is available at github.com/henryqin1997/statem.

ModelReference AccuracyStateM Accuracy
GPT-5.5 xhigh83.1%92.1%
GPT-5.6 Sol Ultra91.9%
GPT-5.6 Sol xhigh95.3%
GPT-5.6 Luna76.7%85.4%
DeepSeek-V4 Flash82.7%88.1%
Source
Hugging Face · Hugging Face Papers
Related news
Research paper
Hugging Face 29 Aug 2026

Hugging Face Audit: 110 of 124 AI Evaluations Fail to Support Their Claims

A new commit-bound census of 124 Inspect Evals units reveals that 110 stop before deterministic inference due to missing historica...

4
Research paper
Hugging Face 29 Aug 2026

Aphanta: New Framework Diagnoses When Image Editing Boosts Multimodal Reasoning

Hugging Face researchers introduce Aphanta, a diagnostic framework that evaluates when image-editing intermediates improve multimo...

5
Research paper
Hugging Face 29 Aug 2026

EditaLive! Enables Real-Time Character Video Editing for Live Streaming

Hugging Face researchers introduce EditaLive, a framework for real-time human-centric video editing in live streams, achieving sta...

4