Researchers have introduced ROWBench, also referred to as PROWBench, a benchmark designed to test whether video models render what a program specifies — the right action, the right outcome, at the right time. The work targets programmable world models, which separate executable dynamics from visual generation and are seen as a promising foundation for next-generation game engines.
According to the paper, existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events. ROWBench addresses this gap with 170 programmatically constructed episodes and 600 proxy videos spanning diverse scenes and interactions.
The benchmark logs entity states and timestamped events — including events outside the camera's field of view — as replayable world records, from which it renders synchronized views and proxy representations. This allows generated videos to be checked against the observable consequences of program execution. An extensible framework constructs scenes, controls behaviors, and can render each camera view in different representations, such as coarse 3D and bounding boxes.
ROWBench covers first- and third-person perspectives, with synchronized multi-view observations available for a subset of episodes. It evaluates entity control, long-horizon memory, and, using two VLM-based metrics — Logic-Render Alignment and Interaction Success Rate — adherence to the prescribed timeline and the visual realization of timestamped engine-recorded events.
The benchmark includes 170 test cases, 200 camera views, and 600 proxy inputs (Coarse 3D, Colored OBB, Semantic proxy), all compiled from engine-recorded world states. It reports 12 metrics, including Logic–Render Alignment, Interaction Success Rate, and Entity Control, grounded in the recorded events, and evaluates current video and world models across urban, indoor, nature, aquatic, aerial, racing, and fantasy worlds.