Researchers at Hugging Face have introduced Second Thought, a training-free inference framework designed to accelerate LLM agents operating in the ReAct paradigm. In this paradigm, agents alternate between reasoning, acting, and observing, but reasoning is traditionally confined to the Thought phase, leaving the action and observation phases idle in terms of cognitive processing.
Second Thought identifies this reasoning idle window—the time spent generating an action and waiting for the environment's response—as an opportunity to run auxiliary reasoning branches in parallel. The framework forks four auxiliary branches the moment a Thought phase concludes, decodes them concurrently with the main loop, and merges the generated thoughts back when the environment observation arrives. This effectively relocates additional reasoning off the main thread's sequential decoding path.
Across three agentic benchmarks and three reasoning LLMs, Second Thought reduced the average turn count in all nine (model, benchmark) pairs and decreased main thread decoding in six of them by up to 43% (roughly 20% on average among those settings). Pass@1 accuracy showed no significant change in seven of nine pairs, with the two significant differences being +12.4 and +10.2 points. Against a compute-matched control that forces an equivalent budget onto the main thread's own reasoning, Second Thought achieved strictly higher Pass@1 with 1.3 to 3.2 less sequential decoding in all four settings where the control applies.
The authors acknowledge that the effectiveness of the idle-window trick depends on the environment call being slow enough to hide the extra compute. They note that the idle window consists of both action generation time and environment execution wait time, and suggest that adaptively issuing side branches based on expected idle windows is a promising future direction.