Autoregressive video distillation has typically relied on multi-stage pipelines based on Distribution Matching Distillation (DMD), where initialization and DMD stages are decoupled and pursue different target distributions. Intermediate results are usually judged by visual scores like VBench. However, a new paper from Hugging Face researchers, titled "DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation," revisits this design from a distributional perspective.
The authors argue that because the distribution matching loss is mode-seeking, a good initialization should match the mode coverage of the target DMD teacher, rather than merely achieving high visual quality. To analyze this, they introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space. This protocol reveals differences hidden by visual scores: some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode-covering ones preserve broader support.
Furthermore, even when target distributions are aligned, the reverse-KL objective of DMD can drive the student toward high-probability teacher regions in late training, reducing coverage and diversity. To address this, the authors propose joint distillation, which combines DMD's mode-seeking objective with a Consistency Distillation-based mode-covering constraint.
Experiments show that the proposed method improves generation quality, coverage, and diversity. Notably, even with a Wan-1.3B DMD teacher, DistillAlign outperforms baselines refined with Wan-14B, underscoring the importance of distributional alignment in autoregressive video distillation. The authors have open-sourced their training data, models at every stage, and full measured intermediate results.