A new critical review published on Hugging Face examines on-policy self-distillation (OPSD), a technique that allows a language model to train on its own generations without requiring a larger teacher model. Instead, the model itself acts as teacher, conditioned on privileged information—such as a reference solution, a plan, or environment feedback—that is unavailable at test time.
The review notes that while OPSD showed early promise, achieving accuracy comparable to reinforcement learning with far fewer generated tokens, it suffers from a dominant failure mode: collapse, the progressive narrowing of reasoning paths the model can produce. Privileged information aggravates this issue, though collapse is not unique to OPSD.
The authors treat collapse as a symptom governed by three levers: (i) where the signal is applied (token weighting), (ii) what the teacher is shown (the nature of privileged information), and (iii) when the signal changes (teacher dynamics and guidance decay). The review restricts its scope to mathematical reasoning, where the method originated and its failure modes are best documented.
No new experiments are reported. The contribution is structural: providing a shared vocabulary for phenomena named differently across papers and drawing a clear line between what is settled and what remains disputed.