Reinforcement learning for code agents typically relies on executable tests that yield binary rewards. Under Group Relative Policy Optimization (GRPO), all trajectories that pass the tests within a rollout group receive identical advantages, leaving no signal to distinguish a clean, targeted fix from one that introduces unnecessary or out-of-scope changes.
To address this, researchers at Xiaomi introduce GAGAR, a framework for quality-aware credit redistribution in code agent RL. Built on dynamic sampling that keeps groups containing both passing and failing trajectories, GAGAR places all trajectories from a group in a shared workspace, where an SFT-trained agentic grader jointly inspects them and ranks the test-passing candidates.
The framework then downweights lower-ranked trajectories and proportionally rescales the advantages of all passing trajectories so their original sum is preserved. This sum-preserving redistribution keeps the relative weights set by quality-based downweighting while shifting credit toward higher-quality implementations.
Three design choices underpin the method: groupwise evaluation, which makes quality differences visible through comparison; an agentic grader that actively inspects intermediate trajectories, patches, repositories, and execution evidence; and a zero-sum redistribution that transfers credit from lower- to higher-quality solutions rather than simply penalizing them.
GAGAR was evaluated at industrial scale using pre-RL SFT checkpoints of MiMo-V2.6-Flash (310B total parameters) and MiMo-V2.6-Pro (1.02T total parameters). Controlled code-only Flash experiments showed improved code agent performance, reduced trajectory-length growth, and more stable training. Large-scale mixed-task RL with both models also showed gains in pass rates and implementation quality, along with fewer interaction turns and lower output token consumption.
The work has been integrated into the large-scale RL training pipeline of the recently released MiMo-V2.6 series. The authors frame GAGAR as a way to scale RL training by allocating additional computation to graders rather than only to model size or rollout volume, and say they are continuing to explore turn-level credit assignment and the application of zero-sum redistribution to different graders, penalty mechanisms, and advantage adjustment strategies.