Physical understanding in AI has long been limited by the gap between pixel-level observations and the underlying mechanisms that govern the world. While modern vision-language models can describe physical events, they often lack explicit representations of object states, physical parameters, and dynamics needed for reliable reasoning. A new paper from Hugging Face, titled "Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning," proposes a solution by representing physical worlds as executable code.
The paradigm, called Code-as-World, expresses physical composition, dynamic evolution, and visual appearance as executable code, providing a compact, quantitatively grounded, and controllable abstraction of the physical world. To construct these representations from multimodal observations—such as natural language descriptions or real-world videos—the researchers developed an agentic discovery loop inspired by abductive reasoning. This loop involves proposing, executing, rendering, verifying, and iteratively refining executable world hypotheses.
As a concrete application, the team used verified executable worlds to provide scalable physical supervision for training vision-language models on quantitative physical reasoning tasks. The resulting model, Code-as-World-VL, achieved state-of-the-art performance on the QuantiPhy benchmark, surpassing leading proprietary models. This highlights the potential of executable world representations as a scalable foundation for physical intelligence.
The authors emphasize that "pixels are evidence of the physical world, not its ontology," arguing that pixel-level observations record appearance but not structure or dynamics. Code-as-World turns raw observations into reusable physical data—explicit states, dynamics, and mechanisms—capturing not only what was seen but the underlying world that could have produced it.