Researchers have introduced Dream-RSI, a framework aimed at making recursive self-improvement practical for autonomous AI agents by rethinking how exploration strategies are evaluated. The work appears as a paper on Hugging Face.
The core problem the authors identify is a dilemma in exploration: fixed strategies cannot adapt as search spaces grow, while online policy optimization demands navigating vast meta-search spaces under delayed and expensive feedback across long-horizon rollouts.
Dream-RSI's key insight is that accumulated discovery history can act as a replay simulator over the realized search space. A lightweight orchestration layer makes exploration explicit and programmable without altering the underlying coding agent. By "dreaming" inside a simulator built from historical discovery trees, the framework obtains immediate, low-cost off-policy feedback to evaluate and refine exploration policies, avoiding repetitive online evaluations.
The improved policy is then redeployed online to drive further discovery, continuously expanding the simulator pool in a self-improving loop. The authors report that across algorithm engineering, mathematical optimization, and GPU kernel engineering, Dream-RSI achieves competitive or improved discovery quality while substantially reducing discovery cost in several settings.
The paper's framing is succinct: to recursively self-improve, agents need to dream — and history can be the world they dream in. Further details are available at dream-rsi.com.