Researchers have introduced a post-training recipe for open-domain text-to-image generation that combines complementary reward signals, arguing that no single reward captures the full range of human preference. The work is described in a paper titled Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards.
The reward system has two parts. A preference reward is trained on large-scale human preference data using a Bradley-Terry objective to capture overall aesthetic and perceptual preferences. Rubric-based rewards explicitly evaluate prompt faithfulness and other desirable properties while providing safeguards against reward hacking.
A central challenge is combining these heterogeneous signals. The authors report that a naive weighted average leads to suboptimal optimization behavior, and they propose a simple reward composition strategy that better balances preference optimization with rubric satisfaction. The approach also introduces anti-reward-hacking gates and combines complementary policies through weight averaging.
In the Arena text-to-image leaderboard, the team's RL-trained FLUX.2-dev achieves an Elo rating 69 points above the base model, while their post-trained Ideogram 4 surpasses every open-source model on the leaderboard with an Elo of 1223.5. The authors note that state-of-the-art claims are based on the Arena leaderboard snapshot as of September 4, 2026.
The paper concludes that effective rewards for frontier generative-model training require broad coverage of user intent and robustness to exploitation under optimization. To support reproducible research, the authors release Arena-T2I-Training, a 1K subset of training data that recovers some gains of full-scale training.