A new research paper from Hugging Face and collaborators introduces Self Gradient Forcing (SGF), a training strategy designed to improve long-video extrapolation in autoregressive video diffusion models. The work addresses a key limitation of the popular Self Forcing approach, where the historical key-value cache is used only as frozen rollout state, preventing future losses from supervising how earlier latents are encoded into memory—a gap the authors call the historical context-gradient gap.
SGF employs a two-pass training scheme. In Pass 1, a no-gradient autoregressive rollout mimics inference, recording both the self-generated context and noisy latents at a sampled denoising exit step. Pass 2 performs parallel context-gradient reconstruction for that step: the generated context serves as stop-gradient clean-latent input, while the model recomputes context KV representations and future-to-context causal attention. This restores the missing memory-writing supervision within the native autoregressive objective.
Extensive experiments on long-horizon frame-wise and chunk-wise tasks under various initializations show that SGF achieves stronger native long-video extrapolation than Self Forcing, particularly in subject identity, background/layout consistency, and temporal stability. Remarkably, using only a 5-second training window, SGF can extrapolate to videos lasting several minutes.
The authors plan to release code and models to advance research on autoregressive video generation. The paper is available on arXiv and Hugging Face Papers.