Hugging Face researchers have released a preprint introducing StarHarness, a framework designed to evolve environment-specific agent harnesses while keeping model weights fixed. The approach addresses persistent model-environment mismatches in tool-rich enterprise tasks by optimizing not the model itself, but the surrounding harness—including prompts, tool interfaces, skills, MCP-backed providers, subagent structures, and agent-loop configuration.
The method constructs a compact evolution pool by stratifying tasks according to baseline failure behavior. It separates proposer-visible search tasks from proposer-hidden selection tasks, and reserves held-out tasks to evaluate generalization. This design aims to prevent overfitting and ensure that improvements transfer beyond the evolution set.
Across three enterprise benchmarks—ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance—harness evolution improved full-benchmark performance by 20–35 percentage points over the default harness after 4–12 accepted changes per environment. The gains persisted on tasks excluded from evolution and transferred without re-evolution across GPT and Qwen model families. Notably, on ITBench, Qwen3.5-27B with an evolved harness outperformed GPT-5.5 on the baseline harness by 19.2 points.
The authors also report a 17–53% reduction in inference cost. Trace analysis attributes the improvements to interface repairs, environment conventions, and operational knowledge that compress search, leading to fewer false-positive diagnoses and shorter trajectories in several settings.
The findings suggest that some of the capability gap often attributed to models may actually stem from how well the surrounding harness fits the environment. With the right harness, a smaller open-weight model can outperform a larger frontier model running the default scaffold.