Researchers behind a new Hugging Face paper, Long-WAM, address a core tension in real-time robot control: robots need enough visual history to infer motion and task progress, but processing that history can slow down action generation. The work proposes a model-system framework for scaling the context of causal world-action models under real-time constraints.
The central finding is that access to history is not the same as using it. Longer histories pay off far more when the video foundation is pretrained autoregressively. Long-WAM first learns causal prediction from robot and egocentric videos without action labels, then preserves this history-to-future structure during world-action adaptation.
On RoboCasa GR-1, increasing context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%. A bidirectionally pretrained initialization shows no net gain. Robot-domain autoregressive pretraining further raises peak success on GR-1 and LIBERO-Long, and Long-WAM achieves the best results among compared methods on LIBERO-Long, RoboTwin 2.0, and DOMINO.
For deployment, streaming observation encoding, asynchronous execution, and hardware-specific acceleration enable operation on RTX 5090, DGX Spark, and Jetson AGX Thor without dropping future prediction. On RTX 5090, each action chunk, including future-video latent prediction, takes 107.4 ms.
Real-time deployment on Unitree G1 and YAM supports dynamic and long-horizon manipulation, including 95% success on dynamic cup stacking, where Pi0.5 and Fast-WAM succeed in none of 20 trials. As a memory-informed executor, Long-WAM also complements higher-level planning in composite tasks.