Hugging Face has published a new research paper, LastOPD: Taming Collapse in Latent On-Policy Distillation, which examines a failure mode in latent on-policy distillation (OPD) when distilling larger teacher models into smaller students.
On-policy distillation corrects a student model on the responses it generates, but its signal is limited to the teacher's next-token distribution. Latent supervision aims to fill this gap by aligning the student's latent states with the teacher's. However, the authors observe two failures when distilling Qwen3-4B and Qwen3-8B into Qwen3-1.7B-Base: early gain followed by late collapse, and better alignment leading to worse behavior.
Specifically, latent supervision alone lifts MATH-500 accuracy from 25 to 46 in 10 steps, but subsequent training degrades performance down to 11 with no recovery. Although the alignment metric steadily improves during this collapse, the most aligned model turns out to be the worst performing. The authors suggest this stems from a mismatch in how the latent signal is applied: layers paired by depth play different roles in the two models, so continued alignment may pull the student toward teacher states it cannot understand.
To address this, the authors propose LastOPD, which applies the latent signal only at the last-layer state—the common interface both language model heads read—and only during a 10-step crossfade into token-level OPD. This preserves the useful part of the latent signal and hands the student to token-level supervision before collapse sets in.
Extensive experiments show that LastOPD improves MATH-500 over token-only OPD by 5.55 and 4.02 points with the 4B and 8B teachers, leads on most held-out datasets, and reaches the final score of token-only OPD in about half the steps. Code is available at https://github.com/Muyiiiii/LastOPD.