Spatial intelligence is more than naming where objects are. Under incomplete observation, a model must recognize what evidence is missing, actively acquire it, interpret it within a shared spatial frame, and then act. Hugging Face researchers introduce VA-Bench to evaluate this complete observe–reason–act–revise loop in embodied settings.
In VA-Bench, general-purpose multimodal large language models (MLLMs) learn procedural context from RGB-only demonstrations, actively select camera viewpoints, issue metric Cartesian commands, and revise those commands based on execution feedback. Crucially, models receive no privileged object poses, oracle trajectories, or learned action heads. A fixed, model-agnostic controller executes only the targets the model specifies.
The benchmark comprises 14 base task families (11 single-arm and three dual-arm), seven held-out geometry/layout variants, and a long-horizon five-object composition track. Twelve primary model conditions were evaluated across three independent runs over the same 20 physically verified seeds per base task, measuring terminal success, nine trajectory-level behavioral diagnostics, and subtask progress.
Results show a sharp gap between perception and action. The best-performing model achieved 100.0% target localization and 78.9% on spatial relations in the annotated run, yet its three-run macro-average task success was only 53.93% ± 3.17%. Active camera control significantly outperformed passive multi-view observation—in one matched comparison, success rose from 27.86% to 57.50%. Held-out geometric transfer reduced success by over 30 percentage points, and no model completed a strict long-horizon episode despite substantial partial progress.
VA-Bench thus probes whether general-purpose MLLMs can convert visual demonstrations and actively acquired evidence into successful embodied action. Code and benchmark are available at github.com/zhangzhongbo2213/VABench.