Coding agents are increasingly being pushed beyond purely digital work into physical-world challenges, especially robotics. A new paper posted on Hugging Face argues that existing robotics benchmarks are too narrow for this shift, since they mainly measure the performance of individual artifacts such as policies or controllers rather than the wider engineering abilities of coding agents.
To address that gap, the authors introduce RLE-Bench, described as a qualifying exam for coding agents acting as robot learning engineers. The benchmark spans four representative robotics development workflows: interactive control, policy learning, perception and estimation, and mechanical design.
Rather than relying on a single score, RLE-Bench uses diverse task-specific metrics to evaluate the artifacts agents submit. These include the success rate achieved by the agents, the policies they train, the harness they build, and the mechanical structures they design. The metrics are aggregated into an overall RLE Index, alongside workflow-specific capability profiles that allow systematic comparison of agents across multiple capability dimensions.
Beyond ranking performance, the paper includes in-depth case studies of agent behavior on representative tasks. According to the authors, these studies highlight both current capabilities and limitations, and point to opportunities that robotics tasks offer for future agent training.
The paper positions RLE-Bench as a benchmark for evaluating coding agents on general robot learning tasks, covering interactive control, policy learning, perception and estimation, and mechanical design.