On-policy self-distillation (OPSD) has become a popular technique for post-training language models, where a teacher model is strengthened with privileged information. However, researchers at Hugging Face have identified a critical flaw: the 'privilege illusion.' In this failure mode, the student learns to rely on privileged signals that are unavailable at inference time, leading to degraded performance despite strong validation results.
In their new paper, the team diagnoses the root cause as an information asymmetry between the privileged teacher and the inference-time student. To address this, they propose Dual-Anchored Policy Distillation (DAPD), a unified framework with two levels of anchoring. Dual-Path Anchoring (DPA) introduces a self-conditioned bridge that aligns reference and rollout behavior along two matched-information paths, preventing the transfer of privilege-dependent behavior. Dual-Source Anchoring (DSA) applies these paths in both directions, reducing reliance on privileged reference guidance while preserving correctness supervision.
Extensive experiments show that DAPD significantly alleviates privilege illusion, outperforming OPSD on Qwen3-4B by an average of +2.00 points across tasks. The gains persist across scales, reaching +2.69 at 4B and +2.78 at 32B. Preliminary results on OLMo also suggest robustness across model families.
The authors note that DAPD is currently formulated for on-policy self-distillation, where teacher and student share the same policy under different information conditions. They plan to explore heterogeneous teacher–student settings in future work, and additional experiments will be included in the next revision.