Hugging Face researchers have published a paper on WorldWeaver (W^2), a streaming multi-agent video diffusion model designed to maintain consistent world states across multiple agents and viewpoints. The model addresses a key limitation of existing autoregressive video diffusion pipelines, which carry forward observation history as conditioning context but struggle to maintain shared state in multi-agent and multi-view settings.
WorldWeaver introduces cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. These registers are grounded with supervision signals spanning individual agent status, global state views (including bird's-eye views), and scene text.
The architecture incorporates a Mixture-of-Transformers design with separate weights for world state modeling and visual frame modeling. Extensive experiments in two-agent Minecraft video generation demonstrate that explicit world-state modeling improves logical consistency and generation quality.