Researchers from Hugging Face have published a new paper, "Dataset Distillation by Influence Matching," introducing a novel approach to dataset distillation that focuses on aligning the final outcome of training rather than intermediate processes. The method, called Influence Matching (Inf-Match), learns a compact synthetic dataset whose effect on the converged model parameters mirrors that of the full dataset.
Traditional dataset distillation techniques often align process surrogates such as per-step gradients or training trajectories. In contrast, Inf-Match introduces a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data. This estimator runs in linear time by unrolling optimization dynamics and applying a first-order Taylor approximation, avoiding costly inverse-Hessian products or convexity assumptions.
The synthetic set is learned by minimizing the mismatch between its influence and that of the real dataset, achieving outcome alignment rather than heuristic process imitation. On standard classification benchmarks, Inf-Match delivers the best accuracy. For example, on Tiny-ImageNet with 10 images per class (IPC=10), Inf-Match attains 31.5% accuracy, a +4.7% improvement over the previous state-of-the-art NCFM.
Beyond classification, Inf-Match scales to vision-language distillation on the Flickr30K dataset. With 200 to 1000 synthetic samples, the method achieves leading average performance on image/text retrieval tasks, outperforming strong process-matching baselines by 2.5% over NCFM. The code will be released on GitHub.