In a new paper, researchers from Hugging Face introduce Explorative Modeling, a paradigm that rethinks how generative models are trained. Unlike traditional approaches that factor the generation procedure, Explorative Modeling factors the training loop by exploring K candidate matches between model generations and data, then training on the best match. This allows predictions to commit to modes rather than blurring them.
The authors show that increasing exploration acts as a third pretraining axis beyond parameters and data. Scaling exploration monotonically improves performance across continuous and discrete domains, including images, video, and language. Notably, gains from exploration increase with scale: improvements climb from 7% to 36% as data scales, and from 13% to 23% as models grow. Efficiency gains more than double at 3x compute.
Concretely, Explorative Models (XMs) achieve 4.1x FLOP efficiency, 6.2x sample efficiency, and 47% better parameter efficiency. They also lift the strongest image-generation recipes to a near-state-of-the-art 1.43 FID on ImageNet without guidance, enable scaling generalization, and allow scaling how end-to-end existing models are.
As a standalone paradigm, XMs enable end-to-end reconstructive generative modeling, matching diffusion on control tasks with 16-256x fewer inference steps. The authors conclude that Explorative Modeling establishes a new pretraining axis for existing generative models and a new end-to-end generative modeling paradigm.