Video object removal is a challenging task that requires not only deleting the target object but also its induced effects, such as shadows, reflections, and lighting changes, while maintaining temporal coherence. Existing methods often struggle with complex real-world scenes involving compositional effects or weakly correlated effects. To address this, researchers from Hugging Face propose EffectLearner, a framework that combines a vision-language model (VLM)-based Object-Effect Reasoner with a Diffusion Transformer (DiT)-based Video Eraser.
The Reasoner analyzes a target-highlighted video using a structured prompt to extract compact effect-aware context, guiding the Video Eraser to perform comprehensive removal. Motion-aware mask guidance and motion-consistency supervision improve coverage and stability under object motion and evolving scene dynamics. This approach enables the model to handle long-tail physical phenomena and dynamically evolving interactions.
To support training and evaluation, the team introduces EffectWorld, a large-scale paired video dataset specifically designed for effect-aware object removal. It covers compositional effects, weak object-effect correlations, long-tail phenomena, and dynamic challenges. A progressive training curriculum combines common supervision with complex-effect data to fully exploit the framework's capabilities.
On the standard ROSE-Bench benchmark, EffectLearner outperforms existing baselines on most metrics and shows clear advantages on both EffectWorld-Eval and the challenging EffectWorld-Wild, demonstrating its ability to deliver high-quality video object removal in complex real-world scenes.