Most egocentric world models predict first-person observations conditioned on a single agent's actions, but real embodied settings often involve multiple agents acting and interacting within a shared environment. A new paper on Hugging Face, Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction, addresses this gap by formulating multi-agent egocentric world modeling as synchronized ego-stream generation for multiple agents interacting through fine-grained actions.
The authors note that existing multi-agent world models rely on coarse actions such as locomotion, camera control, or discrete commands, leaving fine-grained embodied interactions underexplored. Their formulation requires cross-view action consistency, shared-environment consistency, and consistent propagation of interaction-induced state updates.
To meet these requirements, the researchers propose ME-World (Multi-agent Egocentric World Model), which jointly denoises multiple ego streams in a shared token sequence, conditions each stream on all agents' target-view poses, and grounds generation with shared environment memory.
The model is trained and evaluated on both real and synthetic multi-agent data. The team also introduces shared-world consistency metrics covering environment, update, and identity consistency.
Experiments show that ME-World improves shared-world consistency, action control, identity preservation, and video quality over existing methods, according to the paper.