Researchers have introduced HuRo, a pipeline designed to test whether robotized human videos can serve as an effective and scalable source of supervision for vision-language-action (VLA) pretraining. The work addresses the persistent embodiment gap between human demonstrations and robot hardware.
According to the paper, HuRo converts heterogeneous human videos into robot-aligned observations and action trajectories while inferring missing intermediate signals across different annotation levels. This combines visual robotization with motion retargeting to bridge observation and action alignment at scale.
Using this pipeline, the authors constructed the HuRo dataset, comprising about 630K robotized episodes and 142M processed frames drawn from five human-video sources.
Across four real-world manipulation tasks, increasing the amount of robotized pretraining data improved overall task completion from 51.5% to 80.3%, and out-of-distribution (OOD) completion under spatial and visual shifts rose from 34.9% to 72.2%.
Ablation studies indicated that visual robotization improves OOD robustness, and that end-to-end pretraining with retargeted actions outperforms visual-only transfer. A project website is available at 3587jjh.github.io/HuRo.