Researchers have introduced Reflex, a framework that trains humanoid robots to react to moving objects with their whole body in roughly one second — a capability humans exercise effortlessly when returning a tennis shot or catching a falling cup. The work, posted as a paper on Hugging Face, focuses on catching thrown boxes as a representative task requiring three distinct skills: whole-body coordination, real-time reaction to object dynamics, and perception grounded in egocentric vision.
Existing systems, according to the paper, address only part of this problem, often relying on end-effectors, motion capture, or marked objects. Reflex instead decomposes the challenge into three training stages connected by a compact representation of object dynamics. First, a state-based policy learns whole-body catching through reinforcement learning. Second, the policy learns to infer the dynamics of a moving box from delayed and incomplete observations. Third, an egocentric RGB-D encoder learns to recover that dynamics representation directly from visual input while the whole-body controller remains fixed.
The authors argue this decomposition gives each capability a direct training signal and makes visual learning substantially more scalable. In simulation, Reflex achieves 85.7% catch success from egocentric RGB-D, compared with 87.9% when given privileged box observations. Temporal visual history proves critical: removing it reduces success by 35.6 and 39.8 percentage points on centered and laterally offset throws, respectively.
On a real Unitree G1 robot, Reflex catches human-thrown boxes with 65% and 40% success at 2.5 and 5 meters, reaching 76% and 59% of human catchers' success at the same distances.