Text-to-video models have made impressive strides in visual quality, but they often falter when it comes to physical consistency. The challenge lies in the fact that the temporal evolution of a scene must be inferred from a compressed text prompt, leaving the model to implicitly guess the dynamics. Existing chain-of-thought approaches have tried to bridge this gap with intermediate plans or visual states, but these are often non-executable or temporally sparse, limiting their control over the full spatiotemporal process.
To address this, researchers at Hugging Face have developed VideoCoCo, an agentic dual-engine framework that leverages executable Blender code as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly defines the scene and its temporal evolution. This program is then run by a simulation engine to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing.
This decomposition separates process-level reasoning from high-fidelity visual realization, allowing for a more controllable and inspectable generation pipeline. To adapt the video editor to simulated drafts, the team constructed VideoCoCo-3K, a curated dataset of draft-instruction-target triplets.
The results are promising: VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These findings demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.