Researchers have published a new paper on Hugging Face proposing Adaptive Reward Routing, a technique for improving joint audio-video diffusion models through multi-reward guided reinforcement learning.
According to the paper, multi-reward RL can enhance these models along complementary objectives such as modality-specific quality, cross-modal semantic alignment, and temporal synchronization. However, its effectiveness depends on two quantities that shift during training: where reward-driven updates should be applied and how competing rewards should be weighted. Existing approaches reportedly rely on fixed routing and reward weights, which fail to track the model's evolving functions.
The proposed method addresses this through two components. Cross-Modal Influence-Guided Routing localizes updates by using bidirectional cross-attention responses as a proxy for evolving cross-modal influence, dynamically reweighting token-aware losses and scaling gradients across cross-modal layers without additional model interventions. Preference-Preserving Modality-Aware Reweighting coordinates rewards by keeping predefined weights as preference priors and using branch-specific reward-gradient interactions as residual corrections after warm-up, resolving conflicts without letting dominant rewards suppress weaker but essential objectives.
The authors report that extensive experiments show consistent improvements in modality quality, semantic consistency, and audio-video synchronization over strong RL baselines. Ablations and mechanism analyses further validate the complementary benefits of adaptive update routing and reward coordination.