Controllable video generation has long struggled with specifying precise multi-object interactions. Existing methods rely on text prompts or motion-control inputs that constrain pixel movement, but trajectory-based control requires users to draw accurate tracks for multiple objects—a process that scales poorly with scene complexity and becomes ambiguous under occlusion or overlap.
To address this, Hugging Face researchers introduce GraphVid, a graph-conditioned image-to-video generation model that enables interactive control through structured interaction graphs. Instead of tracking pixels, users define relationships between objects via a graph, allowing flexible yet precise multi-subject control.
The team also curated GraphVid-Bench, a large-scale interaction-centric video dataset with structured relational annotations, designed to train interaction-aware video generation models. Despite using substantially less training data and fewer trainable parameters than prior motion-control methods, GraphVid delivers strong controllability and video quality.
In comparisons with Motion-I2V, GraphVid reduces FID by up to 39.9% and FVD by 37.6%, while significantly improving PSNR (from 9.87 to 15.98) and SSIM (from 0.38 to 0.61). The paper, accepted at ECCV'26, highlights the potential of structured semantic interfaces as a powerful paradigm for controllable video generation.