Joint audio-video generation has advanced considerably in recent years, yet existing models still struggle with limited fidelity per modality, weak alignment between text and modality, and insufficient synchronization across modalities. Reinforcement-learning post-training is seen as a promising remedy, but applying it directly to joint audio-video generation is difficult.
According to the paper, three obstacles stand in the way: heterogeneous multimodal rewards entangle learning signals and complicate credit assignment; jointly optimizing two modality towers is computationally expensive because their dynamics diverge; and synchronization evaluation depends on paired samples, which prevents fair reward comparisons.
To address this, the authors propose AV-GRPO, a modality-anchored online diffusion RL framework, together with 5DAV, a decoupled, difficulty-controllable training dataset. AV-GRPO comprises three modules: modality-anchored rollouts that disentangle learning signals and stabilize difficulty; trajectory-locked frozen-tower optimization that lowers cost and reassigns credit; and adaptive objectives and perturbation strengths tailored to modality-specific dynamics.
The approach reframes coupled multimodal preference learning as unimodal subproblems, which the authors say enables more precise reward attribution and better synchronization. The 5DAV dataset decouples samples across five dimensions for systematic training.
Experiments on JavisBench and VABench show AV-GRPO outperforming LTX-2.3 in generation quality, semantic alignment and cross-modal synchronization under both LoRA and full fine-tuning, with ablations supporting the design choices. The authors state that AV-GRPO is, to their knowledge, the first GRPO framework designed for joint audio-video generation models, and that it enables full-parameter or LoRA training of the 22B LTX-2.3 model on just 8 A800 GPUs. Code and data are available at github.com/zhiyuxu03/AV-GRPO.