Researchers have introduced VBVR-Pro, a new closed-loop testbed designed to advance native visual reasoning—a paradigm where images and videos serve not just as inputs or outputs, but as the medium of reasoning itself. The work, released by the Video-Reason team and featured on Hugging Face, addresses key bottlenecks in the field: the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates.
The testbed offers three major contributions. First, it scales task generation by converting visual reasoning into a controlled space of 300 procedurally generated tasks. Models trained on VBVR-Pro demonstrate strong transfer to seven external benchmarks, including RISE-Video, MME-CoF-Pro, and BabyVision, indicating broad applicability beyond the suite itself.
Second, VBVR-Pro provides verifiable reward scorers that are grounded in deterministic, task-specific rules. The authors systematically studied leading multimodal large language models (MLLMs) as judges and identified recurring failure modes in the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers achieve fine-grained alignment with human judgments and serve as reliable reward signals for large-scale multi-task reinforcement learning, leading to stronger post-RL performance.
Third, the testbed enables controlled modality studies across more than 30 image, video, and interleaved generators. The analysis reveals that video generation excels at tasks requiring persistent spatiotemporal state tracking, while interleaved generation offers a compute-efficient alternative. Ablations and probing suggest the existence of vision-native trajectories that are crucial for visual reasoning.
All data, models, scorers, and code are publicly released, with resources available on the project homepage, Hugging Face collections, and GitHub repositories for training and evaluation.