A newly posted paper on Hugging Face introduces SpaceCast-Bench, described as the first benchmark designed to directly and diagnostically evaluate predictive spatial reasoning in vision-language models. Unlike existing benchmarks that mainly test spatial perception — reading off relations already visible in the input — SpaceCast-Bench targets the ability to construct a scene from observations, anticipate how an intervention changes it, and reason about the unseen outcome.
The benchmark is built around an observe-transform-infer framework. It contains 3,862 questions drawn from 182 real-world scenes, spanning 16 task types across three levels: static perception, local prediction, and global prediction. These levels progressively require scene understanding, spatial state updating, and relational inference over unobserved outcomes.
Evaluating 21 models exposed what the authors call a stark gap: the strongest model reaches only 58.0% against 87.2% human performance, while spatially specialized models remain near random chance.
Controlled analyses further found that bridge views are critical for integrating distributed observations, and that explicit 3D evidence benefits models more reliably than generated outcome images or videos. Fine-tuning on the benchmark's programmatically generated data lifted Qwen3-VL-4B from 34.0% to 65.7%, with macro-average gains across six out-of-domain benchmarks.
Code and dataset are available via the project's GitHub repository and Hugging Face dataset page.