Recent research has extended recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, allowing agents to refine their execution mechanisms from experience. However, the paper argues that generalizable harness RSI remains difficult for three reasons: evolving harnesses on evaluation benchmarks or their subsets blurs the line between reusable improvements and benchmark-specific adaptation; single-trajectory updates can conflate systematic harness deficiencies with instance-specific reasoning and solution details; and localizing recurring behavioral deficiencies within monolithic harnesses is hard, while whole-harness optimization can entangle unrelated mechanisms and complicate attribution and validation.
To address these issues, the authors propose ModularRSI, described as a benchmark-disjoint, contrastive, and modular framework for generalizable harness evolution. The approach contrasts successful and failed trajectories for the same task and aggregates evidence across tasks to identify recurring behavioral deficiencies.
ModularRSI decomposes the evolvable harness into five functional modules: Agent Loop, Tool Use, Observation Management, Context Management, and Task Completion Detection. Each module evolves independently within a restricted modification scope, followed by an integration stage that combines the evolved modules into a unified harness and resolves potential conflicts.
To support benchmark-disjoint evolution, the authors curate 2,000 executable evolution tasks from external sources that are disjoint from downstream evaluation benchmarks. Experiments on TB2.0 and SWE-Bench Verified reportedly show consistent improvements on unseen in-domain and cross-domain tasks, with the evolved harness also transferring across different foundation models.
Beyond the framework itself, the paper contributes a benchmark-disjoint evolution set intended to systematically evaluate and compare the generalizability of different RSI methods.