Hugging Face researchers have introduced TimeLens2, a generalist video temporal grounding model that extends multimodal large language models (MLLMs) to not only describe what happens in a video but also identify when supporting evidence occurs. The model predicts a variable-cardinality set of evidence intervals across diverse video lengths, domains, query forms, and viewpoints.
Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization.
The team constructed TimeLens2-93K, a dataset of 93,232 verified grounding instances from 23,793 diverse videos. The pipeline uses caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement to ensure reliable multi-span supervision.
A key innovation is the temporal Wasserstein reward, which computes the exact one-dimensional Wasserstein-1 distance between uniform distributions over merged interval supports. This provides dense, matching-free feedback under unequal cardinalities and equivalent fragmentation, complemented by temporal IoU for precise-overlap feedback.
Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. Specifically, TimeLens2-4B surpasses Qwen3.5-397B-A17B on every benchmark by 7.5 average mIoU points. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.