Hugging Face researchers have introduced a new method called Latent On-Policy Self-Distillation (LOPD) that aims to improve how AI agents learn from their own experience. The approach, detailed in a recent paper, addresses a key limitation in existing on-policy self-distillation (OPSD) techniques, which rely on designer-specified privileged artifacts such as answers, feedback, or skills.
LOPD instead makes the teacher's privileged context itself learnable end-to-end from experience. The system retrieves relevant experiences and composes them into continuous latent tokens that condition a self-teacher, while the student generates trajectories from the task and interaction history, receiving dense token-level supervision at every visited prefix. A privileged-margin objective is introduced to stabilize and regulate the learning of this latent context.
Empirically, LOPD demonstrates strong performance, outperforming RLVR and representative OPSD methods including OPSD, SDPO, and Skill-SD across both agentic tool use and code generation. It also shows high learning efficiency, surpassing GRPO and Skill-SD with less than 30% of their rollout budget. Ablation studies provide direct evidence that making privileged context learnable is necessary for realizing these gains.
The researchers position LOPD as a step toward a more scalable and self-directed paradigm for agent evolution. The code is available on GitHub at https://github.com/bingreeky/LOPD.