Researchers from Hugging Face have introduced DreamX-Phi 1.0, an action-conditioned video world model designed for robotic manipulation. The model takes an observed frame, a language instruction, and a prescribed action sequence—comprising end-effector poses and gripper states—to predict the resulting future observations.
To ensure predictions are faithful, the model incorporates per-arm SE(3) transformations into its attention mechanism using PRoPE-style geometric encoding. This preserves arm identity and rigid-motion structure, preventing issues like moving the wrong arm or losing track of manipulated objects.
For scene-level geometry, the model adds a lightweight depth branch and uses SAM3 masks with a frozen V-JEPA teacher to maintain object consistency during grasping. Additionally, a distribution-matching distillation technique compresses the multi-step generator into a few-step student for efficient deployment.
At the time of writing, DreamX-Phi 1.0 achieves first place on Track 1 and second place on Track 2 of the WorldArena 2.0 Challenge. The model weights and inference code will be publicly available on GitHub after the challenge concludes.