Self-evolving reasoning models improve by learning from questions they generate themselves, but repeated self-training can cause performance to collapse. A new research paper, Questioning the Questions: Sustaining Self-Evolution in Reasoning Models, investigates why this deterioration happens and how it might be prevented.
The analysis points to two recurring quality problems in self-generated questions. First, invalid questions become more common in later rounds, and answer-consistency filtering actually increases their share of the training data. Second, diversity controls based on lexical similarity can miss mathematically equivalent questions phrased differently, leading to a collapse in question diversity in later rounds.
To address these issues, the authors introduce R-Quest, which uses question validity and novelty feedback to guide self-evolution. The solver is first trained to recognize and reject invalid questions; its judgments then shape questioner rewards and filter the solver's training data. To prevent repetition, a frozen base model compares sampled question pairs and provides novelty feedback.
Across 12 benchmarks covering mathematical reasoning, general-domain reasoning, and code generation, R-Quest reportedly achieves the highest average performance across two model families. It also maintains stable gains over ten rounds of self-evolution, peaking in the final round and outperforming R-Zero by 17.32 points.