Diffusion models, while powerful for generative tasks, suffer from error accumulation during iterative sampling—a phenomenon known as exposure bias. A new paper from researchers (affiliated with Sony Research) reveals that this bias has a systematic frequency-dependent component, which can be interpreted as a frequency-dependent signal-to-noise ratio (SNR) error. Crucially, the direction of this mismatch varies across models and timesteps, meaning fixed correction rules do not generalize.
To address this, the authors propose Spectral Alignment (SPA), a lightweight, guidance-based method that calibrates the power spectrum of intermediate predictions to a pre-computed prior. The method operates in two stages: (1) offline fitting of a parametric spectrum model from training data, and (2) inference-time guidance via efficient FFT-based gradient computation.
SPA introduces minimal computational overhead (3-4%) and is complementary to Classifier-Free Guidance (CFG). The researchers demonstrate consistent improvements across diverse architectures, including pixel-space models (DDPM, ADM), latent diffusion models (SD2.0, SDXL), and flow-matching models (SD3.5, FLUX). The implementation is available on GitHub.