Interactive video storytelling lets viewers actively steer how a video unfolds, but once viewers can intervene during generation, a harder problem appears: their requests can carry hidden dependencies on both the grounding source and the video state rendered so far. Those dependencies may never be stated explicitly in any single input and only surface when the source, the rendered history, and the new viewer intent are considered together.
Existing interactive video generation systems tend to focus on following viewer instructions, while source-grounded methods aim to align output with an external narrative or knowledge source. According to the paper, this leaves a fundamental question underexplored: what context should a generation model ground on during interactive continuation, and how can heterogeneous, unstructured inputs be turned into that grounding context?
The authors formulate contextual grounding as transforming heterogeneous input context into an executable constraint model for video generation. Their framework, VIS-Ground, combines three stages: Structured Context Abstraction to recover grounded states and cross-context dependencies; Generation Constraints Induction to project relevant dependencies into candidate-specific constraints; and Constrained Video Generation to enforce those constraints through planning, verification, revision, and rendering.
Across three video generation backbones, VIS-Ground consistently achieves the highest overall composite score, with an average absolute improvement of 10.3 points over the strongest per-backbone baselines. The analysis reports gains in both narrative and knowledge grounding, while flagging remaining challenges in dependency extraction and in faithfully realizing constraints during video rendering.
More details are available on the project page at vis-ground.github.io.