Hugging Face has released a new research paper introducing Video-DeepResearch (Video-DR), a framework designed to push multimodal agents beyond static images and into the complex realm of continuous video streams. This new setting demands dense spatiotemporal grounding combined with open-web exploration, a challenge that current models struggle to meet.
The paper identifies two critical bottlenecks in existing agents: modality bias, where agents skip visual tools in favor of textual search, and parametric knowledge leakage, where models rely on internal memory instead of genuine tool-augmented execution. To counter these, Video-DR employs a decoupled perception-exploration pipeline with stage-wise tool unlocking, compelling the agent to perform exhaustive cross-frame visual grounding before any web retrieval.
The training recipe is a two-stage process: supervised fine-tuning followed by Group Relative Policy Optimization (GRPO). This approach enables autonomous exploration, breaking the imitation-learning ceiling and allowing agents to discover tool-use patterns rather than merely copying them. The framework also introduces Video-DR-Bench, a human-AI collaborative benchmark with 200 complex, multi-hop VQA instances.
Empirical results show that the Video-DeepResearch-35B-A3B model achieves a state-of-the-art average accuracy of 64.0%, surpassing proprietary models like Claude-4.5-Sonnet (59.0%) by 5.0 points, and significantly outperforming GPT-5 (52.5%) and Gemini 2.5 Pro (57.5%). A smaller 30B-A3B variant reaches 59.3%, competitive with Claude-4.5-Sonnet, demonstrating the effectiveness of the training paradigm even at compact scale.
The code is available on GitHub, and the authors envision a future of streaming deep research—'know everything in vision.'