General-purpose agents have become adept at writing code, using tools, and completing complex digital tasks. A new research paper posted on Hugging Face asks how far those capabilities carry over into the physical world, introducing RobotWorld, a simulation testbed designed to evaluate robot use.
According to the paper, RobotWorld contains 84 tasks spanning manipulation, mobile manipulation, locomotion, driving, and aerial control. The benchmark converts instructions and observations into physical task execution through robot interfaces, and it imposes explicit interaction budgets alongside executable success checks.
By analysing task outcomes together with execution traces, the authors identify which capabilities transfer from digital settings and which gaps prevent reliable completion. They report that current agents can build sophisticated perception and control workflows, including image segmentation, camera calibration, spatial estimation, and dynamics-based computation.
Yet those abilities do not consistently compose into successful behaviour. The paper notes that agents lose task-relevant object states even after reaching commanded poses, fail to correct ineffective actions, recover too late, or mistake unfinished tasks for completion.
The uneven transfer also varies by model: Astra succeeds more often on spatial and constrained-contact goals, while Opus 5.5 succeeds more often on continuous-balance and timed-interaction goals. The authors position RobotWorld as both a rigorous proving ground and an empirical account of remaining capability gaps, offering concrete targets for training and designing more reliable physical-world agents.