Reinforcement learning has become a key tool for optimizing video generation models, and a reliable reward model is essential to that process. However, existing video reward models often produce unstable scalar scores because they map complex and subjective video quality directly onto a single number without any explicit evaluation criteria. This causes scalar drift, where the scoring scale collapses or shifts across different prompts, making the reward unreliable for reinforcement learning.
To address this, researchers introduce RewardVerse, a rubric-based video reward framework inspired by professional human annotation engineering. Rather than scoring videos directly, RewardVerse generates explicit evaluation criteria as a dynamic rubric that acts as an intermediate representation between the evaluation query and the scorer. This rubric-guided scoring provides a stable semantic anchor that mitigates scalar drift.
To train this collaborative pipeline efficiently, the authors propose Rubric-Guided Policy Optimization (RGPO), a two-stage algorithm. In the first stage, RGPO warms up the scorer using self-evolving seed rubrics. In the second stage, it jointly optimizes the rubric generator to produce query-adaptive evaluation criteria while continuously aligning the scorer with human ratings.
Experiments on the 16-dimensional EvalVerse benchmark and external datasets show that RewardVerse reduces scalar drift and achieves state-of-the-art performance on both pointwise and pairwise evaluation. According to the authors, the framework also delivers a robust and interpretable reward signal for reinforcement learning in video generation.