Researchers from Hugging Face have unveiled SimWAM, a novel World Action Model (WAM) designed for end-to-end autonomous driving. Unlike existing WAMs that require costly future frame generation at inference time, SimWAM leverages video generation purely as a training signal, enabling a lightweight and efficient planner.
The model co-trains a pretrained video expert and a lightweight action expert using joint flow matching. An isolated attention mask ensures action prediction remains independent of future frames, allowing the video branch to be discarded after training. This leaves a self-contained planner that directly predicts trajectories, significantly reducing inference latency.
SimWAM's architecture is modular: the two experts share no parameters and interact only through a unified attention interface. This design allows the video backbone to be replaced or the action expert scaled independently without altering the learning objective or inference pipeline. Additionally, reinforcement learning is applied to optimize a compositional driving reward beyond simple trajectory imitation.
In evaluations, SimWAM achieves a 91.5 PDMS score on the NAVSIM benchmark, surpassing state-of-the-art WAM-based planners with substantially lower latency. It also demonstrates zero-shot transfer to nuScenes, highlighting its robustness and generalizability. The code and model weights are publicly available on GitHub.