Researchers at the Hong Kong University of Science and Technology (HKUST) have introduced Hierarchical Self-Improvement (HSI), a framework that enables a single, frozen large language model (LLM) to autonomously evolve the executable harness that surrounds it for specific tasks. Unlike traditional approaches that treat the harness as a fixed artifact after deployment, HSI treats it as a living component that can be rewritten and improved over time.
The framework operates across three hierarchical scopes: a task harness that executes tasks, an evolver that rewrites the harness, and a meta-evolver that refines the evolver's strategy code. A key design choice is a thinking-on/off mechanism that disables reasoning during task execution but enables it during self-modification, isolating the contribution of harness evolution from the model's own reasoning.
In experiments using the BALROG benchmark with DeepSeek-V4-Flash-Preview as the frozen backbone, HSI achieved consistent improvements on moderate-difficulty tasks: +39.3% on BabyAI, +33.0% on Crafter, +25.0% on TextWorld, and +15.0% on MiniHack (raw % Progress). The framework also demonstrated strong held-out generalization on unseen BabaIsAI sub-suites, with best-test scores of 0.98 on BreakStop and 1.00 on GoTo from a 20% unseen split.
However, the authors identify two critical bounds: a feedback-fidelity bound, since evolution requires informative reward signals, and a backbone capability bound, as harness redesign cannot overcome the frozen model's limitations. On tasks beyond the backbone's capability (e.g., NLE), harness evolution provided no improvement, underscoring the empirical limits of this approach.
The code is available on GitHub, and the paper is published on arXiv. This work highlights a viable axis for improving frozen LLM agents through task-specific harness evolution, with clear boundaries for when it works.