On-policy self-distillation (OPSD) trains a language model using a frozen copy of itself as a teacher — a copy that is given extra information such as the correct answer or a worked solution. Intuitively, handing the teacher more of the solution should give the student more to learn. A new paper posted on Hugging Face asks how much that privileged information actually contributes once distillation itself is accounted for.
To separate the two effects, the authors built AMPLE-Math, a reusable suite of 5,319 mathematical problems with six reasoning views that all share the same answer. Each view is compared against a matched reference-free distillation baseline, allowing the researchers to isolate the value of the privileged reference.
Using a thinking-enabled teacher supervising direct-response rollouts, the study finds that reference-free distillation already explains much of the improvement seen in Qwen3-1.7B under thinking-enabled evaluation, both in-domain and on external benchmarks. Evidence of an additional benefit from the reference was modest in Qwen — strongest for a polished solution — while complete traces added two percentage points in SmolLM3-3B at step 50.
Crucially, those gains depend on how the student is trained. At the same checkpoint, swapping short direct-response rollouts for long thinking-enabled rollouts turned gains into losses in both model families, even though the problems, references, and evaluation were held fixed. Teacher profiles and matched loss interventions in Qwen further showed that changing token-level supervision can leave student behavior largely unchanged.
The authors conclude that OPSD may improve access to reasoning capabilities the model already has, through parameters shared by direct-response and thinking-enabled inference. The value of a privileged reference, they argue, lies in what it adds to this cross-mode transfer — not in how much of the solution it reveals.