Training tool-use agents to improve from their own experience has long been a challenge in AI. Supervised fine-tuning relies on fixed teacher-distilled trajectories, while reinforcement learning with sparse rewards provides weak supervision for long-horizon interactions. A new paper from Hugging Face introduces DeepSearch-Evolve, a self-distillation framework that addresses this gap by leveraging a deterministic and verifiable environment called DeepSearch-World.
DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks. The environment supports reproducible search and page-reading tools, enabling key agentic cognitive behaviors such as progress verification, grounded reflection, and failure recovery. These features are essential for self-evolving agents.
The DeepSearch-Evolve framework iteratively performs trajectory generation, filtering, data mixing, and fine-tuning. Without distillation from more capable models, the resulting DeepSearch-World-9B model achieves competitive performance: 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA. These results demonstrate that verifiable environments enable scalable self-evolution for long-horizon web agents.
The authors plan to release the environment, the 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.