Continual post-training poses a dual challenge: models must acquire new capabilities while retaining previously learned knowledge. Self-Distillation Fine-Tuning (SDFT) addresses this by using self-supervision with in-context demonstrations, but it depends on strong in-context learning and instruction-following abilities — capabilities that smaller models often lack.
According to the paper, this limitation means SDFT fails on the smaller models that are critical for low-latency, on-device use. The authors trace the root cause of this failure, showing that poor acquisition and forgetting correlate linearly with the post-trained model's distribution shift. This, they argue, reveals the necessity of in-distribution learning.
To address the problem, the researchers introduce Synthetic Self-Distillation (SynthSelf). The method imitates SDFT's in-distribution rewrite by outsourcing the task to a minimally larger, same-family aligner. Combined with off-policy supervision and rejection sampling, SynthSelf provides the high-quality supervision small models need without breaking their original distribution.
Evaluations spanning 4 acquisition datasets (including a mixture) and 14 retention benchmarks show that SynthSelf consistently outperforms 6 methods — including off-policy and on-policy distillation — across 3 small model scales. The authors state that extensive further analyses confirm SynthSelf paves the way toward effective small-model continual learning.