Spatial reasoning has long been a weak point for vision-language models (VLMs), largely because RGB images do not directly supply geometric evidence. Prior approaches either inject 3D information at inference time, incurring architectural and latency costs, or rely on outcome-based rewards that supervise only the final answer.
A new paper proposes GPD (Geometry-Privileged Distillation), which treats geometric evidence as the "privilege" in on-policy self-distillation (OPSD). The authors argue that spatial errors often begin in perception: a misjudged depth or direction can only be corrected using the scene's true geometry, which is already available in the 3D-scanned sources of spatial training corpora.
In GPD, depth, semantic, and bird's-eye-view (BEV) cues are rendered as compact text and routed to the teacher model alongside the reference answer for each question. A privileged KL term, applied only to incorrect trajectories, augments GRPO training, while the deployed model remains RGB-only at inference.
On a 4B backbone, GPD achieves 57.1 on VSI-Bench and a 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD on spatial reasoning benchmarks. Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restricting distillation to incorrect trajectories.
Code, model weights, and a dataset are released publicly: GitHub, GPD-4B model, and GPD-15k dataset.