Researchers have introduced Neighborhood On-Policy Self-Distillation (N-OPSD), a method that improves mathematical reasoning models by drawing supervision from a pool of complementary teacher experts rather than a single fixed teacher. The work appears as a Hugging Face paper and targets the standard on-policy self-distillation (OPSD) setup, where a privileged teacher that sees a reference solution supervises student-sampled prefixes.
The core observation is that small perturbations to the teacher's parameters yield experts offering complementary, reference-aligned corrections at different positions. Their combined pool covers more useful positions than the unperturbed privileged teacher. N-OPSD turns these corrections into supervision at states the student actually visits.
Offline, the method uses greedy selection to build a compact pool of frozen experts, rewarding filtered reference-token gains beyond the pool's current best at each position. Online, routing separates the anchor direction from its level of support: MaxPeak selects the anchor token, while a quantile rule chooses among experts whose top token matches it. The student then learns from the chosen expert's full next-token distribution using the clipped forward-KL objective inherited from OPSD.
Evaluation covers AIME 2024, AIME 2025, and HMMT February 2025 across Qwen3-1.7B, 4B, and 8B. Averaged over three independent runs per method, N-OPSD improves the three-benchmark Average@12 over OPSD by 2.75, 1.67, and 1.94 points for the three model sizes, respectively. The authors report gains across all nine model-size–benchmark combinations.
Ablations support filtered reference-token gains as a selection criterion, and student-prefix continuations indicate the pool can be used beyond the reference trajectories used for selection. Accounting for overlap within the pool and routing by state further improve student accuracy. At inference time, only the distilled student is required.