On-policy distillation (OPD) is a technique for transferring teacher capabilities by supervising trajectories sampled from the student's own policy. However, its generalization behavior remains poorly understood, as most studies evaluate OPD on a single domain and on benchmarks close to the training data.
In a controlled study, researchers varied one generalization factor at a time, from in-domain distribution shifts to cross-domain transfer and the multi-teacher setting. They found that OPD transfers a teacher's reasoning behavior rather than its answers to particular problems: training difficulty barely matters, and even problems the teacher never solves are useful.
Transfer depends strongly on the origin relationship between teacher and student: same-origin pairs bring the student close to the teacher across languages, reasoning horizons, and even other domains, whereas cross-origin pairs mostly fit the trained distribution. This broad reach is a double-edged sword: since routing prompts to domain experts cannot confine each teacher's influence, combining them yields a mixture-dependent seesaw among their capabilities.
These results clarify when OPD generalizes and offer a useful perspective for diagnosing multi-teacher OPD.