Camera-conditioned world models generate interactive videos where commanded actions should produce expected scene changes while maintaining coherent appearance, geometry, and temporal dynamics. Existing reward models assess these requirements separately: geometry-based rewards estimate trajectory execution but ignore visual quality, while image-based rewards measure frame quality without capturing action execution or temporal dynamics.
To address this gap, researchers propose WorldReward, a vision-language model (VLM)-based pairwise preference reward model that unifies action-consistency and visual-quality evaluation. WorldReward decomposes paired videos into action-aligned chunks, organizes each chunk into structured visual evidence, and aggregates chunk-level decisions by voting into separate video-level action and visual-quality preferences.
Training WorldReward required constructing a large-scale reasoning-augmented preference dataset, generated using structured judgments from a frontier VLM and refined through tool-based agent auditing and targeted human review. The team also introduced WorldReward-Bench, a human-annotated benchmark measuring reward-model agreement with human preferences across action consistency, appearance quality, and motion quality.
According to the paper, WorldReward achieves the highest agreement on all three dimensions, exceeding GPT-5.5 by 3.42, 1.45, and 3.56 percentage points, respectively. When used for reinforcement learning (RL) post-training of HY-WorldPlay 1.5, it consistently improves both action execution and visual quality across short- to long-term horizons.