Multimodal large language models (MLLMs) have made significant strides in perception tasks, yet they often stumble when faced with complex visual cognitive challenges that require multi-step temporal reasoning. According to a new research paper from Hugging Face, this limitation stems from the inherent ambiguity of language-based reasoning, which struggles to accurately articulate continuous visual transformations.
To address this, the researchers propose ChronoVision, a novel multimodal framework designed to align visual logic with latent imagery. During supervised fine-tuning, a Reconstructive Visual Head predicts the latent representation of the final transformed state, while an ROI Attention Locating module focuses the model on key visual evidence via semantic span queries.
In post-training, ChronoVision applies reinforcement learning with an implicit process grounding mechanism, guided by a composite reward function that evaluates outcome correctness, latent process alignment, and unsupervised visual focus. This multi-faceted approach ensures the model not only gets the final answer right but also follows a coherent reasoning path grounded in visual evidence.
To rigorously evaluate temporal tracking, the paper introduces Vbvr-VQA, a novel dataset that reformulates video reasoning into a strict image-ordering task. Experiments demonstrate that ChronoVision achieves state-of-the-art performance on Vbvr-VQA with 74.8% in-domain and 71.6% out-of-domain accuracy, alongside a strong 55.0% accuracy on IntPhys2, a highly challenging cross-domain benchmark.