Hugging Face has released a new research paper introducing JoyAI-Video-Edit, a 16-billion-parameter autoregressive diffusion framework designed for real-time, open-ended video editing. The model operates without access to future frames or a predefined video duration, enabling low-latency causal generation with bounded computational resources.
The framework integrates three key innovations: chunk-wise autoregressive adaptation for streaming, Source-Anchored Distribution Matching Distillation (SA-DMD) to preserve source fidelity during two-step generation, and Long-Horizon Autoregressive Distillation to mitigate accumulated temporal drift. These components collectively reduce train-inference mismatch and ensure long-term temporal consistency.
According to the paper, extensive automatic and human evaluations show that JoyAI-Video-Edit substantially outperforms existing streaming editors and remains competitive with strong offline systems on both short and long videos. The complete system achieves end-to-end 720p video editing at approximately 30 FPS on a single Nvidia B200 GPU.
The code is publicly available on GitHub, allowing researchers and developers to explore and build upon the framework.