In the rapidly evolving field of AI, agents learn by interacting with their environments. However, most environments are hand-built and static, remaining blind to an agent's weaknesses and quickly becoming obsolete as the agent improves. While recent environment generation methods aim to address this, they often require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments.
To overcome these challenges, researchers from Hugging Face propose EnvHarness (Environment Harness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness can be applied across diverse domains while ensuring that every reshaped environment retains its original verifier.
To automate this process, the team introduces EnvRigger, which treats the target policy as a black box. EnvRigger observes execution trajectories to synthesize EnvHarness components that target diagnosed flaws, validating them through fresh rollouts. This enables continuous, targeted co-evolution of the policy and its environment, providing a superior optimization signal for reinforcement learning.
In experiments across five benchmarks in four domains, EnvHarness outperformed both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. The code and webpage are publicly released at envharness.com/code.