While coding agents have made significant strides in modifying existing codebases—such as bug fixing and feature implementation—constructing a complete program from scratch remains a formidable challenge. Even frontier models resolve fewer than 1% of tasks on ProgramBench, a benchmark for from-scratch program synthesis. A key obstacle is the lack of scalable training environments that span the entire software engineering life cycle, as existing frameworks typically focus on a single phase.
To address this gap, researchers at Hugging Face introduce MindForge, an automated pipeline that converts open-source command-line programs into source-free environments, exposing only a compiled reference executable and its documentation. This approach enables the construction of training environments from repositories disjoint from those in ProgramBench, ensuring clean evaluation.
Using MindForge, the team built 562 cleanroom environments across six compiled languages and collected 1,001 whole-life-cycle trajectories from a teacher agent (GLM-5.2), averaging 181 turns and 177K tokens each. These trajectories were refined via infrastructure-noise recovery and reasoning rewrites to produce clean supervision.
Fine-tuning Qwen3.6-27B on these trajectories resulted in MindForge-27B, which improved its ProgramBench average test pass rate from 37.98% to 49.51%—a relative gain of over 30%—surpassing DeepSeek V4 Pro and reaching the performance band of GLM-5.1 and Claude Opus 4.7. The model outperformed its base on 152 of 200 tasks, indicating broad gains rather than isolated successes.
Remarkably, MindForge-27B generalized to seven unseen benchmarks, including long-horizon repository generation and translation, bug fixing, feature implementation, and cross-language issue resolution. Absolute gains include +31.00 points on RepoZero-C2Rust, +14.16 on DeepSWE, +10.70/+4.56 on NL2Repo-Bench (with/without tests), +5.04 on SWE-bench Verified, +5.93 on SWE-bench Pro, +5.22 on SWE-bench Multilingual, and +4.94 on FeatBench—all statistically significant.
Behavior analysis revealed that the model learned to genuinely engineer: it works roughly twice as long per task while making proportionally fewer errors per command, and nearly doubles the rate at which reasoning and failure recovery translate into actual code edits. This work represents the first data pipeline targeting the end-to-end software engineering cycle.