Hugging Face researchers have unveiled LiveAnimate, a novel system that enables real-time, long-form pose-driven human animation using a 14-billion-parameter video diffusion transformer. The system is designed to synthesize videos of a target person from a single reference image and a driving pose stream, a task that traditionally requires minutes to hours per clip with diffusion-based approaches.
To achieve real-time performance, LiveAnimate employs a two-stage training pipeline. First, it adapts a pretrained bidirectional diffusion transformer into a block-causal autoregressive generator via Reference-Anchored Teacher-Forcing Adaptation. Second, it reduces the sampling budget to just three steps through Block-wise Self-Forcing Distillation.
A key innovation is Pose-Retrieval Sink Attention (PR-Sink), a bounded KV-cache mechanism that preserves appearance over extended streams. It combines a Static Sink (anchoring the first generated block), a Dynamic Sink (holding a pose-retrieved historical block), and a three-slot Rolling Window. When a pose recurs, PR-Sink restores relevant appearance context without retaining the entire sequence, keeping memory and per-block latency constant regardless of stream duration.
Combined with Ulysses sequence parallelism and operator fusion, LiveAnimate achieves 19.63 FPS streaming inference on two NVIDIA H100 GPUs. On a three-minute benchmark, it maintains nearly constant perceptual quality and identity from the first 30 seconds to the final minute, while prior systems degrade substantially or require hours of offline computation.