Reinforcement learning from verifiable rewards (RLVR) excels when tasks have programmatic checkers, but most long-horizon agent domains lack such ground-truth signals. In this outcome-blind setting, multi-criteria rubrics offer a popular alternative, yet scoring them once per trajectory yields a single scalar that poorly reflects performance across tens of steps.
To address this, researchers from IBM and Hugging Face introduce DRACO (Distributing Rubric-based Advantage for Credit Optimization). DRACO generates rubrics dynamically during training to match the policy's evolving capabilities, scores each completed trajectory once, and redistributes that judgment over the responsible steps to produce differentiated per-step advantages in GRPO. The redistribution is closed-form, requiring no trained attribution module.
On the AppWorld benchmark, DRACO improves the base model by 15.9 points and surpasses GRPO trained with sparse ground-truth rewards by 5.3 points, all without using any verifiers. On out-of-domain Tau-Bench, it gains 5.3 points over the base model even without a frontier judge, outperforming both ground-truth-reward training and other rubric-based approaches.
The code is available on GitHub at https://github.com/IBM/draco.