Real-world software development is rarely a solo endeavor. Users often inspect and modify code while an AI coding agent is working, yet most benchmarks evaluate agents in isolation. A new framework called SWE-Touch, introduced by Hugging Face researchers, aims to change that by stress-testing agents in shared workspaces.
The framework injects validated Counter-Edits—small, plausible code changes that conflict with the task—at critical moments during a repair trajectory. These edits are accompanied by contextual user messages, simulating a user actively touching the code. The researchers mined task-critical regions from multiple repair trajectories and used a separate User Patch Generator to construct the edits.
Evaluating nine frontier coding models on SWE-bench Verified, the team found that Counter-Edits lower the average resolve rate by 7.7 percentage points, and the degradation persists on longer-horizon benchmarks like SWE-Bench Pro and DeepSWE. Notably, 63.3% of failed runs simply retained the conflicting code untouched, indicating a lack of awareness of the evolving workspace.
Only Claude Opus 4.8 and GPT 5.5 showed strong resilience, while open-source models that score competitively on autonomous benchmarks degraded substantially—up to 16.5 points. Ablations confirmed that the code edit itself, not the accompanying message, drives the performance drop.
The findings suggest that optimizing for static leaderboard performance does not ensure robustness in collaborative settings. Agents need to detect workspace changes, reconcile conflicting edits with the task, and re-validate affected behavior—capabilities that current models largely lack.