CLI-based software-engineering agents have matured rapidly, yet the open ecosystem has converged on a single training environment: trajectory datasets used to fine-tune open models are collected almost exclusively under OpenHands. Models fine-tuned on this data score well under OpenHands but degrade substantially when deployed under any non-training scaffold. Untrained base models do not show this divergence, indicating the gap is fine-tuning-induced and tied to the conventions of the training scaffold.
The authors argue that a load-bearing scaffold-specific behavior is planning structure, in two senses: explicit planning (a pre-execution plan produced as a first-class artifact) and implicit planning (the structural conventions that shape execution throughout the agent loop). Under this hypothesis, closing the gap requires moving planning from a fixed scaffold artifact to a learned model capability.
To test this, they introduce Decoupling CLI Agent Scaffolding (DCAS), a backend-substitution interception layer that routes API traffic between any CLI scaffold and any backend model without modifying the scaffold. This enables cross-scaffold evaluation and planning-aware trajectory collection. Using DCAS, a controlled plan-source intervention confirms planning quality is a high-leverage component, with gains exceeding the cross-scaffold drops observed.
Across four top open models at the 30B to 32B scale on SWE-bench Verified, every fine-tuned model degraded on at least one scaffold it was not trained on, one losing 29 points under mini-swe-agent and another falling to 8.4%. The untrained base models did not show this spread. Changing only who wrote the plan moves Qwen3-Coder-30B-A3B-Instruct from 42.8% to 57.8% Pass@1 with the executor and scaffold held fixed.
Full-parameter SFT on just 576 planning-aware trajectories collected through DCAS reaches 55.8% under self-plan, then 57.2% on a Claude Code release published after collection, 49.4% on OpenCode and 42.4% on mini-swe-agent, none of which the model saw during training. The two senses of planning are empirically separable in training data.