Researchers have unveiled DiagEvo, a novel approach to language-model self-evolution that derives training direction from the model's own failure history. The method addresses a key limitation in self-play paradigms, where solvers often plateau or decline without external guidance.
DiagEvo introduces a diagnostician that extracts recurring error causes from a solver's failures and stores them in a hierarchical error-cause memory. This memory groups related causes under skill nodes, tracking each as Active or Mastered based on self-consistency on targeted questions. The challenger then uses this information to balance targeted problem generation with free exploration.
A key innovation is double-confidence filtering, which retains intermediate-difficulty questions only when the most common solver answer has a clear vote lead, ensuring reliable pseudo-labels. This allows the system to generate a curriculum entirely from self-play data, without needing human examples or external documents.
In evaluations, DiagEvo outperformed all baselines across nine benchmarks for three different solvers, including Qwen3-4B, Qwen3-8B, and OctoThinker-8B. On Qwen3-8B, it achieved 72.3% mean accuracy on mathematical reasoning benchmarks, 4.5 points above R-Zero, and 57.4% overall, 1.1 points above DARC.
The method also showed that longer problems don't necessarily lead to better learning. While free exploration saw problem length balloon from 45.6 to 168.9 words by round five with performance plateauing, DiagEvo stabilized length around 73 words and continued improving for five rounds. The paper is available on arXiv, with code forthcoming.