Looped Transformers are often assumed to reason better simply by iterating more, but a new research paper argues the opposite can happen. As the number of loop iterations grows, reasoning accuracy can fall: noisy state updates overwrite correct intermediate deductions and can even undo solutions that were already complete.
The authors trace the problem to how recurrent state is carried between loops. Once an error appears in an early iteration — often through long-range propagation across the recurrence — later iterations must recover lost information from an already degraded representation, which they struggle to do.
Their proposed fix, InfiLoop, is a loop-native residual connection that learns which past computations to retain and how much of each new update to accept. It combines content-based weighting with a learned temporal decay to maintain a running summary of recurrent states, and uses an exact streaming recurrence so its persistent aggregation memory stays constant as the loop count grows. The adaptive update is designed to suppress unreliable proposals while preserving useful intermediate states.
In experiments across reasoning tasks, a 7M-parameter InfiLoop model is reported to outperform existing recursive architectures, reaching 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2. On Sudoku-Extreme, the model reportedly keeps improving with test-time looping beyond 20,000 effective steps, suggesting added depth translates directly into stronger reasoning. The authors note the approach is not best everywhere — on Maze-Hard it is roughly on par with prior work — and frame the depth scaling as the central result.
The paper's code is available at github.com/pixeli99/InfiLoop.