Loop engineering is emerging as a practice for organizing development work around coding agents. Instead of writing each prompt by hand, practitioners design loops that monitor progress, assign work, run checks, and decide what the agent should do next. However, even with a capable coding agent, a loop may trust a stale progress note, skip needed verification, spend its budget in the wrong direction, or stop before the task is safe to submit.
To address the challenge of evaluating loop guidance independently from the coding agent's ability, researchers introduce LoopArena, a benchmark that measures how well one model can guide a separate coding agent through a long-running task. The model under evaluation is the Controller, which after each coding round receives a structured summary and instructs a fixed coding agent, the Worker, on what to do or verify next, or decides whether to stop.
LoopArena evaluates this ability in three complementary settings: Type I scores next-step Loop Contract selection through execution-validated questions without running the Worker at evaluation time; Type II executes repeated control over a selected slice of a full task; and Type III evaluates the paired full task from its original state.
Results show that on full tasks, the best observed Strict Success Rate is 24.69%, leaving substantial room for improvement in long-horizon loop control. Across Controllers, the paired reduction in estimated inference cost averages 64.4%, and Type II produces a similar ordering under the main Core criterion (Spearman's ρ=0.9747). The benchmark data and evaluation code are released at https://github.com/AMAP-ML/LoopArena.