Vision-language models (VLMs) are increasingly deployed in embodied agents, where they must interpret visual inputs, reason about spatial relationships, and make task-level decisions. However, a fundamental mismatch persists: general VLMs excel at high-level reasoning but often miss critical visual details, while specialist vision models capture those details but cannot translate them into task-level decisions.
To bridge this gap, researchers from Hugging Face propose SpatialCLI, a framework that teaches VLMs to reason with spatial tools and progressively internalize the specialist perceptual capabilities those tools provide. The approach unfolds in three stages: Call, Learn, and Internalize.
- Call: Exposes specialist vision models as spatial tools to augment the VLM's perception.
- Learn: Uses Cold-Start Supervised Fine-Tuning (SFT) and agentic reinforcement learning to improve tool use.
- Internalize: Verbalizes successful tool-use trajectories, allowing the VLM to internalize specialist capabilities for tool-free inference.
The team also introduces SpatialCLI-Bench, a benchmark comprising 516 examples for compositional perception across localization, segmentation, depth, and pose estimation.
On the MindCube benchmark, SpatialCLI elevates Qwen3-VL-8B-Instruct from 29.3% to 84.6% accuracy when tools are available, surpassing GPT-5.6 Sol with tools (72.1%). Remarkably, after internalization, the model retains 73.8% accuracy even without tools, demonstrating the effectiveness of the framework in transferring specialist capabilities into the model itself.