Researchers have introduced BVB (Blender-VideoBench), a benchmark that evaluates agentic video understanding by requiring multimodal agents to programmatically reconstruct real-world videos as animated Blender scenes — without relying on diffusion models. The work is presented as a Hugging Face paper.
The premise is direct: if an agent genuinely understands a video, it should be able to rebuild it in code. To keep comparisons fair, every agent programs its reconstruction through a lightweight harness called Mini-BVB, operating in an identical sandbox under a shared cost limit.
Each reconstruction is rendered from its animated camera and scored on two axes. Dual VQA measures how many spatiotemporal facts the reconstruction preserves, while Latent Similarity measures perceptual closeness to the source video. The overall score is a square-root mean, which rewards balanced performance across both dimensions.
The benchmark spans 288 real videos and 51 agent configurations drawn from 10 model families, with analysis covering semantic retention, perceptual similarity, reasoning effort, and cost. The best configuration reached 88.6 Latent Similarity but retained only 53.7% of the source-correct spatiotemporal answers.
Adding reasoning effort improved visual similarity but did not close the gap in factual accuracy. In a blind study with 15 raters across five configurations, Latent Similarity correlated strongly with human preference. The authors conclude that programmatic reconstruction is a viable test of agentic video understanding, and that semantic retention remains the central challenge.