Modern AI agents rely on sophisticated inference harnesses—such as Claude Code, Codex, and OpenClaw—to manage multi-turn reasoning, tool use, and external system access. However, these complex harnesses make it difficult to train agents end-to-end using open infrastructure, as standard SFT/RL stacks cannot natively handle stateful, multi-process harness inference.
To bridge this gap, Hugging Face researchers present OpenForgeRL, an open-source framework for training harness-based agents end-to-end in diverse environments. The framework uses a lightweight proxy that intercepts model calls from the harness, recording them as training data for a standard RL codebase (e.g., veRL). A Kubernetes orchestrator runs each rollout in its own remote container, enabling scalable training on any harness in any environment.
OpenForgeRL decouples training from inference, allowing researchers to train and improve agents directly in the real harnesses and environments they will be deployed in. The framework was validated across complex harnesses and environments, including tool/claw-based agents and multimodal GUI browser- and computer-use agents.
Key results include: OpenForgeClaw achieves 31.7 pass^3 and 55.9 pass@3 on ClawEval, and 33.7 on QwenClawBench. OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. These results outperform open baselines of similar size on nearly all benchmarks, and in the GUI setting match or surpass models several times larger.
The analysis reveals that harness choice (e.g., ZeroClaw, OpenClaw, Codex) significantly impacts learning difficulty, and that RL improves agentic reliability—such as self-verification, tool coverage, and completing multi-step plans—though critical abilities like error recovery remain weak.