Reference-to-video (R2V) generation is moving toward a more general paradigm known as omni R2V generation, in which models must handle increasingly versatile reference control. According to a new paper posted on Hugging Face, existing benchmarks have not kept pace with these capabilities: their test cases cover a limited range of reference types and compositions, and their evaluation protocols tend to measure holistic reference consistency rather than whether individual reference factors are properly preserved, disentangled, and routed.
To close these gaps, the authors introduce OmniVBench, a benchmark that broadens R2V evaluation across more reference types, fine-grained control tasks, and richer reference compositions. It covers 7 task families and 18 fine-grained tasks spanning content, motion, style, structure, narrative, and multi-reference settings.
The benchmark also proposes factor-grounded evaluation using 12,172 case-specific checklist items. This protocol assesses whether intended reference factors are faithfully preserved, correctly disentangled and bound to their targets, and properly realized according to the instruction.
Alongside the benchmark, the team releases the Omni-R2V Dataset, described as industrial-grade training resources for diverse R2V tasks. Drawing primarily on a large-scale corpus of professional video footage, it contains 340K processed training samples spanning diverse reference types and multi-reference compositions. The authors develop task-specific pipelines for reference-target pair construction, offering a practical and scalable recipe for omni R2V data construction.
Extensive evaluation of advanced open- and closed-source R2V models reveals clear performance gaps across task families and evaluation dimensions on OmniVBench, highlighting remaining limitations of current R2V models.