A newly posted paper on Hugging Face introduces Negative Self-Distillation (NSD), a framework aimed at improving the reasoning abilities of large language models (LLMs) without relying on ground-truth solutions or an external teacher.
The work addresses a known weakness in On-Policy Self-Distillation (OPSD), a popular self-improvement paradigm in which a model acts as its own teacher using privileged information such as correct answers. According to the paper, OPSD can seriously harm performance on complex reasoning tasks: by forcing the student to imitate an artificially confident reasoning trace conditioned on privileged information, it suppresses expressions of uncertainty and penalizes the exploratory, self-corrective behavior needed for difficult problems.
NSD takes the opposite approach. Instead of imitating privileged solutions, it optimizes the model to diverge from flawed reasoning. The model itself generates a question-specific negative condition — for example, acting as a “careless reasoner” — and the student's distribution is pushed away from this self-generated negative teacher.
The authors note that naively applying unlearning objectives to achieve this divergence is problematic, because flawed reasoning tokens are confounded with basic linguistic tokens; penalizing both indiscriminately risks catastrophic degradation of foundational language capabilities. To resolve this, they design a dynamic gating mechanism that automatically identifies and isolates reasoning-critical tokens, so gradient updates target only behavioral flaws while preserving the model's linguistic priors.
Empirically, the paper reports that NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning baselines. The framework is described as fully self-bootstrapped and free of any need for ground-truth solutions or an external teacher.