On-policy distillation (OPD) is a promising technique for transferring reasoning abilities from stronger teacher models to smaller student models. However, applying OPD to long-context reasoning teachers and short-context students poses practical challenges, including tokenizer mismatches, distribution shifts, response length explosion, and training instability.
In a new paper, researchers present SimpleOPD, a method that addresses these issues by performing distillation in a shared text space and aligning only tokens that occupy identical text spans under both tokenizers. This tokenizer-agnostic approach enables effective knowledge transfer even when the teacher and student use different tokenizers.
To prevent excessive generation length and frequent truncation, the method introduces a student reference KL loss and masks the advantages of special termination tokens such as </think> and <|im_end|>. This constrains the student from drifting too far from its initial policy, mitigating teacher-student distribution mismatch and fostering steady length growth.
Experiments on both same-family and different-family student models, including Qwen3, Qwen3.5, Intern-S2, GLM-4.7, and Gemma-4, show consistent gains in mathematical reasoning, especially in natural-language math proving. Notably, Intern-S2-Preview improves by 21.2 points on ProofBench, reaching 55.2 and surpassing Gemini-2.5-Pro. The method also improves performance on science benchmarks such as HLE and HiPhO, suggesting that OPD transfers reasoning capabilities that generalize beyond the mathematical training domain.