A new research paper from Hugging Face introduces Latent Dynamics Reasoning (LDR), a video world model designed to learn the underlying laws of motion from pixels alone. Unlike leading video diffusion models that focus on generating visually plausible frames without explicitly modeling temporal transitions, LDR casts latent transitions as explicit kinematic integration, regressing only higher-order residuals to drive rollouts.
The model operates on a structured latent space rather than dense convolutional features, which improves its ability to extrapolate. The researchers validated LDR on a controlled white-box physics benchmark (PhyWorld) covering five tasks: uniform motion, parabola, collision, bouncing, and looming, with a focus on out-of-distribution scenarios.
Results show that LDR's gap between in- and out-of-distribution error is over 20 times smaller than a video diffusion baseline, under both single- and joint-task training at 256² resolution. This performance comes with 26 times fewer parameters and 143 times faster inference.
Notably, LDR can generalize under severe distribution shifts. For example, trained only on red balls moving left-to-right, it correctly predicts the motion of a blue square moving right-to-left. The authors claim this is the first video world model to extrapolate learned dynamics beyond its training distribution.
Resources: Paper | Project Page | Code | Model | Data