Researchers have published DiffusionART (Diffusion Reinforcement with Aggregation-First Reward Tilting), a new algorithm for fine-tuning diffusion models with black-box rewards. The work targets a trade-off between two existing families of methods: reverse-process approaches such as Flow-GRPO, which are limited by poor credit assignment, and forward-process approaches such as DiffusionNFT, which are efficient but unstable without carefully tuned exponential-moving-average (EMA) off-policy training.
The authors argue that NFT-style objectives contain a hidden forward-reverse consistency force that joint positive-negative training only partially cancels, and that EMA stabilizes training through restoration rather than off-policy rollouts. DiffusionART instead aggregates each prompt's rewards into a shared reward-improving direction for reverse-process states, removing that consistency force by construction, and adds a trust-region-masked second update for drift-adaptive restoration without EMA tuning.
The method requires no differentiable reward model, remains effective with fewer samples per prompt, and supports pairwise reward models through a variant called DiffusionART-Pairwise. Across image and video generation, the authors report efficient and stable training without per-task tuning.
On HunyuanVideo-13B, with equally weighted visual and motion quality rewards, DiffusionART reportedly reaches the late-stage mean reward levels of DiffusionNFT and Flow-GRPO-Fast using 12.02 and 11.72 times fewer GPU hours, respectively, while achieving 28.8% and 53.0% higher mean training rewards.
In a discussion thread, the authors contrasted their approach with Finite Difference Flow Optimization (FDFO). They noted that FDFO relies on SDE sampling and finite differences between nearby points, whereas DiffusionART computes Cov(x₀, r) without a proximity requirement, describing the finite-difference construction as a special case of their formulation. They also said FDFO appears to need four updates per rollout versus two for DiffusionART, and that FDFO has not yet been validated on video generation.