Hugging Face has published the technical report for StepAudio 3 Music, a large-scale model for long-form music generation that supports explicit musical planning and open-domain text control. The work is presented as a research paper on the Hugging Face papers platform.
The system rests on a discrete-continuous design. The StepAudio Music Tokenizer represents audio as a 50-Hz stream drawn from a 65536-entry single codebook, trained with semantically informed self-supervised and multi-task objectives to preserve musical structure and reconstruction-relevant information. A flow-matching diffusion Transformer (DiT) then predicts continuous StepAudio VAE latents, which the VAE decoder converts into 48-kHz audio. The authors report that this design was guided by comparisons of single-codebook VQ, Semantic and Acoustic RVQ, and different DiT configurations.
For explicit planning, a Mixture-of-Experts autoregressive model uses ABC notation to produce an intermediate arrangement plan, termed ABC-CoT, before predicting music tokens. This makes harmony, rhythm, and melodic structure part of the generation context. A progressive training curriculum and supervised fine-tuning support song and instrumental generation, accompaniment generation from dry vocals, and cover-song synthesis for clips up to 5 minutes and 30 seconds.
With reinforcement learning via direct preference optimization (DPO), the final model achieves the highest AudioBox Content Enjoyment, Content Usefulness, and Production Quality scores, as well as the highest MuQ-MuLan similarity among the evaluated systems, alongside competitive SongBench results. On the preliminary Artificial Analysis Music Arena Vocals leaderboard, it records a Quality Elo of 1105, behind only Suno V5.5 and Mureka, and ahead of Suno V5, MiniMax models, and other systems.
Audio demonstrations are available at the project page linked in the report. The paper also appears alongside related 2026 work on full-song generation, audio language models, and controllable text-to-music systems.