Robot manipulation policies need to know which objects matter and where they are in space, but the pretrained backbones behind today's robot foundation models — from language in vision-language-action models (VLAs) to video generation in world-action models (WAMs) — do not explicitly require this metric grounding. As a result, it is typically learned implicitly from robot demonstrations. A new paper on Hugging Face proposes Grounded Action Models (GAMs), a paradigm of robot foundation models built around 3D grounding.
GAMs can be conditioned using language, points, or box prompts. These inputs are first transformed into a shared object-centric representation of the selected objects, capturing target-focused visual features and metric object geometry. That representation is then mixed with robot state history through a multi-stream transformer to predict action chunks.
Beyond autonomous operation, GAMs can act as a low-level controller directed by a high-level planner through its various input modalities, enabling long-horizon and memory-dependent manipulation.
On RoboTwin 2.0, GAM reports an average success rate of 55.3% across 50 tasks, compared with 52.0% for Spatial Forcing. Under scene randomization it reaches 47.6% versus 30.4% for Abot-M0, despite its action policy being trained only on clean-scene demonstrations. On LIBERO-PRO, it reports a state-of-the-art average success rate of 61% across 16 perturbation settings, compared with 53% for π0.5, with the largest gains when targets are relocated or newly designated.
On real hardware, GAM retains 17/20 successes under visual shift on a bimanual YAM, versus 4/20 for π0.5. When composed with a Molmo2 planner on a Franka, it achieves 64.7% in-distribution and 49.8% out-of-distribution step completion on long-horizon and memory-dependent tasks.