Most video editing systems still lack explicit layered video representations, which limits their ability to perform realistic compositing, object reuse, and consistent manipulation. This is especially true for video object insertion and layer decomposition, where current methods rely on implicit inference or per-scene optimization due to the absence of explicit foreground-layer supervision.
To address this, researchers at Hugging Face introduce TriLayer, a large-scale triplet video dataset containing aligned composite, background, and foreground videos. The foreground layers include both object appearance and associated visual effects, providing explicit supervision that enables models to learn layered video representations directly rather than inferring them implicitly.
Building on this dataset, the team proposes DBL-Diffusion, a dual-branch diffusion framework that jointly models RGB composites and RGBA foreground layers through shared denoising and cross-branch interaction. The framework is instantiated in two tasks: DBL-Insert for layered object insertion, which generates explicit RGBA layers for realistic compositing and flexible post-editing, and DBL-Decompose for video layer decomposition, which recovers foreground and background layers using triplet supervision.
Experiments demonstrate that explicit layer modeling substantially improves both insertion fidelity and decomposition quality, marking a significant step forward in video editing technology.