Researchers have introduced Hebero (Heterogeneous Benchmark for Robot Learning), a GPU-parallel benchmark built on Isaac Lab that supports joint training and evaluation of a single policy across 40 heterogeneous manipulation tasks. According to the paper, existing benchmarks rarely combine GPU-parallel simulation scale with heterogeneous manipulation tasks and standardized multi-task reinforcement learning evaluation.
Scaling experiments reported in the work show that increasing the number of parallel replicas per task improves success rates under a fixed wall-clock budget. To address sparse rewards and limited demonstrations, the authors propose Demonstration-Guided Policy Optimization (DGPO), which reuses demonstrations for dense tracking rewards and asymmetric value learning. Its shared stack is designed to support controlled comparisons of learner-specific demonstration interfaces within PPO.
Within the DGPO framework, the team introduces IW-ABC, which combines a lightweight per-task learning progress signal to coordinate adaptive behavior cloning (ABC) — relaxing demonstration guidance as task progress increases — with importance weighting (IW) that emphasizes lagging tasks in PPO updates.
With 50 demonstrations per task, IW-ABC achieves 90.1% state-input mean success, outperforming the strongest baseline, FAMO-ABC, by 7.8 percentage points. Its visual counterpart reaches 93.5% mean success. Real-world experiments further demonstrate that a single multi-task policy trained in simulation can successfully perform four tasks on a physical Piper robot.
The project page is available at hebero-rl.github.io.