Researchers have introduced AlayaVista, a camera-controllable streaming video world model that separates panoramic scene evolution from perspective video synthesis. The design targets a persistent tension in interactive world models: maintaining broad scene context under camera motion while still producing high-fidelity observations with low latency.
According to the paper, existing methods face a representation trade-off. Perspective models operate on local views and must preserve off-screen content across long rollouts, while wider spatial coverage usually comes from synthesizing full-sphere video or building explicit 3D representations. AlayaVista instead draws on the complementary roles of global context and selective local acuity in visual perception.
Given a single perspective image, the system first constructs a 360-degree scene prior using a pretrained panorama expansion model. It then evolves the scene as a camera-conditioned panoramic latent state. A latent viewport renderer maps that state to the requested perspective video latents, and a perspective refiner restores detail, suppresses artifacts, and performs super-resolution.
For efficient streaming, the team adapted the panoramic generator to chunk-autoregressive generation and distilled both panoramic generation and perspective refinement into few-step processes.
To supply the supervision this design requires, the authors built MUGEN, described as a large-scale real-world panoramic video dataset containing 1,318 hours of video at resolutions of at least 4K, along with rich semantic and geometric annotations.