Hugging Face researchers have introduced ZimaBlue, a scalable framework designed to learn generalizable World Action Models (WAMs) from large-scale video. The work addresses a key challenge in robotic manipulation: the scarcity and cost of action-labeled robot trajectories, which limits the diversity of physical experience available for training.
ZimaBlue leverages egocentric videos as a scalable source of embodied experience, capturing object interactions, contact dynamics, tool use, and long-horizon behaviors across diverse environments. The framework employs a three-stage training curriculum: causal embodied video pre-training on large-scale human and robot videos, video-action mid-training with a unified action representation to ground visual dynamics in robot trajectories, and final specialization to a target robot for deployment.
To enable real-time control, ZimaBlue adopts an asynchronous Slow-Fast dual-system architecture. A high-capacity Slow world model provides generalizable spatiotemporal representations, while a lightweight Fast branch enables 30 Hz action prediction on an NVIDIA RTX 4090.
In real-robot zero-shot evaluations, scaling from target-robot data alone to over 120,000 hours of embodied video improved success rates from 36.1% to 77.8%. The framework also delivered strong performance across multiple benchmarks, with particularly pronounced gains on unseen tasks, suggesting that video scaling is a practical route toward more generalizable robot control.