In a new research paper, Hugging Face challenges the prevailing approach to scaling world models, which relies on training on ever-larger crawled video datasets with increased compute. The authors argue this strategy is inefficient, as it lacks grounded reward signals necessary for effective reinforcement learning (RL) post-training.
The paper draws a parallel with the success of code agents, where executable code allows compilers and runtimes to provide high-quality rewards for RL. In contrast, spatial generation currently depends on fuzzy proxies like CLIP scores, which are biased and difficult to use for RL. Game development, the authors posit, offers a missing reward environment: a scene encoded by a game engine is an executable world specification, enabling the engine to check collisions, physics, navigability, and bounded playability, while the developer provides global verification by judging scene acceptance.
To capitalize on this, the paper introduces Reinforcement Learning with Human-Engine Verification (RLHEV), a post-training paradigm that combines dense engine signals with implicit human acceptance feedback from the development process. This approach also provides real-world long-horizon trajectory data for RL post-training, addressing both the reward and data challenges.
The paper was shared on Hugging Face and has attracted recommendations for related work, including benchmarks for game development agents and embodied reasoning, highlighting the growing interest in agentic approaches to world model training.