Researchers have introduced 4Director, a video world model that conditions generation on an explicit 4D scene representation. Rather than relying on ambiguous 2D image-plane cues or incomplete 3D tracks and blobs, the method reconstructs each object once from the input image as a canonical mesh and moves it with one prescribed rigid transformation per frame.
This representation provides an intuitive 3D control interface and prevents unobserved geometry from being regenerated independently in every frame. The controlled scene is rendered as a depth video, and a Motion Adapter transforms this geometric scaffold into video while synthesizing view-consistent appearance, illumination, and non-rigid dynamics.
To support training, the team constructed RealCOD-Rigid, a new dataset of 20,774 clips annotated with rigid 3D scenes by an automatic pipeline. They also introduced Identity-Gated IoU (IG-IoU), a metric that jointly evaluates adherence to prescribed object motion and preservation of object identity.
According to the paper, experiments show that 4Director consistently outperforms prior methods in visual quality and in camera and object control. The authors describe the approach as turning an image into an editable 3D scene: every object becomes a complete mesh placed in the same 3D space as the background and camera, allowing users to place the camera, draw rigid trajectories, and bring in new objects from other photos.
Unlike 2D boxes or 3D blobs, a complete mesh already contains each object's hidden side, so it is not reinvented frame by frame when the object turns or the camera orbits. Interactive 3D scenes and comparisons are available on the project page.