Large language models (LLMs) are increasingly used to write and repair production code, but a new study from Hugging Face researchers reveals a systematic flaw: deletion avoidance, the tendency to retain code that an intended edit requires removing. This behavior can lead to codebases that are harder to maintain, even when patches pass tests.
The study, titled "To Add Is Machine, To Delete Is Human," analyzes the performance of leading models on the SWE-bench Verified leaderboard. It finds that even on tasks all five top models solve, deletion recall against the developer patch reaches at most 71.7%. Models locate the correct file for over 92% of required deletions but cut the exact line in under 52% of cases. Instead, 29.0% of passing patches wrap the targeted code in a guard or fallback, a pattern the researchers call Guard-and-Go.
To measure this phenomenon, the team curated CanItDelete, a benchmark of 200 tasks mined from real commits where the entire required edit is deletion. Results show that even the best model, Claude Opus 4.8, fails 21% of tasks, while GPT-5.6 Sol fails 26%, and smaller open models fall to 18.0% success. When tests were retrofitted to fail if targeted code remains, resolution rates dropped from 63.2% to 41.9% across four frontier models.
The paper also explores mitigation. Providing exact lines to delete helps but introduces over-deletion. A pilot study shows that adding just 0.7% deletion-focused data during post-training reduced incomplete deletion by 13.9 points and improved SWE-bench Verified by 5.3 points, suggesting the behavior is undertrained rather than beyond reach.