In a new research paper, Hugging Face investigates the scaling properties of text conditioning in visual generation, a topic that has been largely unexplored due to the assumption that diffusion loss does not scale with the number of tokens in natural-language prompts. Surprisingly, the authors find that the converged diffusion loss does scale with the amount of structured language in the prompt.
To quantify structured language, the researchers adapt two complementary measures: a white-box likelihood metric (GPG) and a black-box attribute metric (ED). Across controlled training runs, they observe that the converged diffusion loss decreases approximately linearly with GPG and follows a power law with ED.
Guided by these scaling properties, the team improves diffusability by constructing structured prompts with semantic and geometric annotations derived from images, and improves promptability by training a prompter through supervised fine-tuning, cold-start, and verifier-gated on-policy distillation.
The resulting system outperforms all evaluated open-weight models on nearly every compositional, reasoning, and world-knowledge benchmark, while matching or surpassing the strongest closed-weight models on most evaluations.