Hugging Face has introduced HumanTracker, a comprehensive benchmark and preference-aligned metric designed to evaluate humanoid motion tracking with a focus on perceptual quality and physical contact stability. The work addresses a critical gap in current evaluation methods, which often rely on kinematic errors that fail to capture the physical artifacts most noticeable to human observers.
Traditional metrics average per-frame pose differences but overlook issues like foot skating and mistimed touch-downs, which are crucial for stable and natural motion. Moreover, existing test suites are small and lack the diversity needed to stress contact-rich, long-horizon behaviors. HumanTracker aims to make evaluation both perceptually aligned and scalable.
The benchmark includes approximately 153 hours of optical motion trajectories from multiple professional performers, organized into four motion families with text labels for fine-grained diagnosis. Alongside the benchmark, the authors propose HumanScore, a preference-aligned metric trained on 12K motion pairs containing 24K motions.
In tests across representative state-of-the-art trackers, HumanScore better predicts human preferences and reveals contact and stability failures that kinematic metrics often miss. The goal is to move beyond simple kinematic errors and evaluate what truly matters: stable, natural, and physically plausible motion.