Researchers at Hugging Face have published a paper detailing a novel approach to data design for generalist image generation models. The work, titled "From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation," addresses a key limitation in conventional pipelines that optimize task-specific datasets in isolation.
The proposed framework introduces a capability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling. It employs three specialized yet interoperable data engines to build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association. Caption experts align T2I and editing supervision across tasks and granularities.
A multi-stage curriculum jointly evolves task composition, visual-concept distribution, data quality, and image resolution along the dependency order of capability acquisition. Capability-aware evaluation closes the loop through targeted retrieval, expert construction, and gap-aware resampling.
At scale, the framework curates a 440M-image T2I corpus, 120M editing pairs, and over 27M image-entity pairs. Using this infrastructure, the researchers trained multimodal diffusion models at two scales from scratch, with 3B and 6B parameters respectively. Quantitative evaluation on CPI-Bench and qualitative evaluations across diverse text-to-image and editing scenarios demonstrate broad visual coverage, versatile rendering, and effective transfer across generative capabilities.