Joint video-audio generation models have advanced rapidly, but optimizing them with reinforcement learning (RL) has been hampered by fragmented reward signals. Existing methods combine separate metrics for audio quality, visual fidelity, and synchronization, which fail to capture the holistic semantic and temporal coherence that shapes human preferences. This can lead to reward hacking, where models score high on metrics but produce incoherent or unfaithful content.
To address this, researchers from ShareLab-SII introduce VA-Judger, the first reward model specifically designed for joint video-audio generation. It uses a chain-of-thought reasoning approach to jointly assess visual and audio quality, text alignment, audio-video synchronization, semantic coherence, and overall human preference. The model is trained on a new large-scale human-preference dataset, VAPref-10K, containing 9K prompts and 10.3K fine-grained paired comparisons from open-source generation models.
The team also presents VA-Judger-Bench, a benchmark with both in-domain and out-of-domain model comparisons to evaluate whether reward models truly align with human preferences. Their training pipeline first learns from pairs with clear quality gaps, then distills reliable preference explanations for harder near-quality comparisons via rejection sampling, and finally performs dimension-wise RL that decomposes human feedback into individual quality dimensions for denser reward signals.
Experiments show that VA-Judger substantially outperforms single-dimensional metrics and omni-modal baselines like Qwen3-Omni, and generalizes reliably to unseen closed-source generation models. When used to post-train LTX-2, the resulting model achieves a 62.30% human preference rate, compared to 27.63% for the OmniNFT-trained version and 10.08% for the original LTX-2. It also achieves the best performance on 11 out of 13 objective metrics.
The project, paper, code, models, dataset, and demo are publicly available. Further training code and the full VAPref-10K dataset will be released soon.