Videos captured through glass often suffer from reflections that degrade visual quality and hinder downstream vision tasks. While single-image reflection removal has been extensively studied, video reflection removal remains largely underexplored due to the lack of paired video data, temporally coherent removal models, and dedicated evaluation benchmarks.
To address this, researchers from Hugging Face and collaborators have introduced a closed-loop framework that unifies physics-grounded reflection simulation, diffusion-based video dereflection, and benchmark evaluation. The framework comprises three key components: S2R-Synthesis, S2R-Removal, and S2R-Bench.
S2R-Synthesis generates paired reflected and reflection-free videos by performing physics-grounded augmentation in the structure space and rendering realistic reflected videos with a trained video diffusion renderer. The augmentation models key glass-related effects, including roughness-induced blur, thickness-induced ghosting, and reflectance variation.
Based on the synthesized data, S2R-Removal is introduced as the first diffusion-based video reflection removal model. It adapts a pretrained video diffusion prior through reflection-aware latent adaptation and one-step pixel-geometric refinement, recovering the clean transmission in a single denoising step.
Furthermore, S2R-Bench is built as the first benchmark for video reflection removal, supporting both full-reference evaluation and real-world human perceptual assessment. Experiments on S2R-Bench and multiple public image benchmarks demonstrate state-of-the-art performance and faster inference than even non-diffusion baselines, validating the effectiveness of S2R-Synthesis.
The project page is available at https://codingwzp.github.io/VideoDereflection_S2R. Code and pretrained weights will be released soon.