Hugging Face has released a new research paper introducing 4DAnyone, a framework that reconstructs 4D humans from a single uncalibrated monocular video. The method generates multiview-consistent videos and lifts them into 4D Gaussian Splatting (4DGS), enabling high-quality novel-view synthesis and downstream reconstruction.
The core challenge addressed is the bounded-attention-context problem in video diffusion models. When generating tens of target views required for 4DGS, existing models split them into groups, leading to two bottlenecks: reference-context complexity grows linearly with the number of views (O(N)), weakening cross-view guidance, and disjoint target groups fail to exchange information, causing structural drift.
4DAnyone introduces two complementary designs: Reference Context Packing (RCP) compresses growing reference views into a fixed-length mixed-resolution context with O(1) complexity, and Target Context Routing (TCR) rotates target-view groupings during denoising to share context at high-noise steps and stabilize details at low-noise steps.
The team also built the MVGameHuman dataset using an in-house game engine, combined with light-stage and in-the-wild video datasets for training. Experiments on DNA-Rendering and DyMVHumans show that 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction, with robust in-the-wild generalization.
For more details, visit the project page: https://4danyone.github.io.