Reward models are a central tool for evaluating and improving visual generation systems, but most existing approaches map a task condition and a candidate output directly to a single scalar score. As a result, the criteria behind each judgment remain implicit rather than stated.
In a new paper titled Think Before You Score: Thinking Reward Model for Visual Generation, researchers propose a different paradigm: explicitly determining what matters for each individual case before judging how well the candidate performs. Following this principle, they introduce the Thinking Reward Model (TRM), which formulates case-adaptive rubrics, carries out rubric-guided assessment, and produces fine-grained pointwise rewards.
The authors also report that conventional pairwise preference optimization can induce score polarization, and they propose Pairwise Dual-Group Relative Policy Optimization (PD-GRPO). This method uses pairwise supervision to sharpen reward discrimination while preserving fine-grained pointwise scoring.
According to the paper, extensive experiments on image generation and editing reward-modeling benchmarks show TRM achieving state-of-the-art performance among open-source reward models, while remaining highly competitive with proprietary alternatives. The authors further state that using TRM as a reward for reinforcement learning consistently improves diverse visual generation models, indicating that its fine-grained, case-adaptive rewards translate into effective optimization signals.