Reinforcement learning (RL) for coding agents often depends on long-running agent harnesses that manage tool integration, repository contexts, and execution feedback. However, the native execution environments of these harnesses are inherently misaligned with policy-gradient training: environmental crashes and reward hacking can corrupt outcome signals, while train-inference discrepancies decouple rollout behavior from policy updates.
To address this, researchers from Hugging Face present LEGO-RL, a framework that bridges native coding-agent harnesses with scalable policy-gradient optimization without modifying their internal control flow. LEGO-RL is built on three pillars: faithful optimization via in-process LLM proxying, reliable execution via scalable sandbox orchestration, and observable training through an integrated monitoring plugin with a Live UI.
The framework was evaluated by training the sparse Mixture-of-Experts (MoE) model Qwen3.5-35B-A3B using GSPO across three native coding-agent harnesses. On SWE-bench Verified, LEGO-RL improved performance from 64.0% to 70.4% on OpenHands SDK, from 62.4% to 68.2% on Claude Code, and from 57.2% to 66.6% on OpenCode, while maintaining a rollout-training probability correlation above 0.99.
LEGO-RL is open-source, with all code, data, and models available on GitHub and Hugging Face.