Researchers from Hugging Face have unveiled GameWAM, a unified world-action model designed for native video-game control. Unlike existing agents that map visual and task context directly to actions without explicit world dynamics, or interactive world models that predict visuals but lack policy capabilities, GameWAM jointly generates future visual observations and executable keyboard-mouse trajectories. This is achieved through parallel visual and action generative processes using block-causal conditioning and flow matching.
To support joint world-action learning, the team constructed synchronized gameplay and GUI trajectories. GameWAM handles heterogeneous native controls by predicting a gameplay/GUI mode at each action step and generating actions with mode-specific prediction distributions and continuous-action normalization. This allows the model to manage continuous camera or cursor motion alongside discrete controls within a single Action-DiT flow.
For long-horizon interaction, GameWAM employs block-cycle control: it predicts beyond the committed horizon, executes only a short action prefix, and replans from new observations. A bounded within-cycle KV cache and hierarchical cross-cycle visual history preserve temporal continuity under bounded memory, making the approach scalable for extended gameplay sessions.
Experiments across Minecraft and ViZDoom show that GameWAM achieves competitive task success while using fewer executed native actions than compared agents in Minecraft across all evaluated task categories. The research also uncovers a phenomenon termed Low-Frequency Action Source Imprinting (LASI), where low-frequency components of the sampled action source systematically steer coarse camera motion, potentially leading to persistent directional bias when reused across replanning steps.
The project page is available at https://yunncheng.github.io/GameWAM/.