Researchers from KAIST AI and Kakao Corp. have introduced GRACE (Generation-Aware Latent Compression for Efficient Video Generation), a two-stage framework that compresses pretrained video autoencoders to accelerate video diffusion models. The work addresses a key bottleneck: while highly compressed autoencoders reduce the token count for Diffusion Transformers (DiTs), they often degrade reconstruction quality and shift the latent distribution away from what the DiT was trained on, forcing costly retraining or adaptation.
GRACE solves this by keeping a frozen base latent from the pretrained encoder and learning a residual latent to capture information lost under stronger compression. Crucially, the compressed latent is aligned with the pretrained latent in the feature space of the frozen DiT, so the autoencoder is optimized for generation rather than reconstruction alone. This preserves compatibility with the pretrained DiT.
After compression, the DiT undergoes lightweight fine-tuning with asymmetric denoising, where the base latent is denoised ahead of the residual. According to the paper, GRACE reduces the token count of Wan2.1-I2V-14B by 8x and its latency by 11.1x at 480x832x81 resolution, while matching the generation quality of the original pretrained pipeline on VBench.
The project page and code are publicly available, with extensive qualitative results. The work was conducted while the KAIST AI authors were interns at Kakao Corp.