Long-horizon LLM agents solve tasks through many reasoning-and-action turns but receive only a single verification signal at the end. Because deployed agents typically face streams of related tasks, their own execution trajectories are a natural resource for improvement — yet in-context adaptation methods store reflections, memories, or skills as text, making reuse dependent on retrieving the right experience and on a frozen policy executing it.
A new research paper introduces Online Agentic Test-Time Training (OaTTT), in which the LLM's weights are trained on its own execution trajectories during deployment. The agent executes each task once, in a single pass over the stream, and that executed trajectory plus its verification result is the only learning signal for weight updates that persist across tasks. The authors note that directly imitating or reinforcing the generated tokens of this single attempt destabilizes the policy.
To address this, the paper proposes ASCENT (Agentic Self-distillation for Cross-task EvolutioN at Test-time), which self-distills verified experience instead. A stable version of the LLM — its frozen initial copy — receives the verified trajectory as privileged information and predicts next-token distributions along it with this hindsight. Distilling those distributions into persistent LoRA fast weights updates the agent for later tasks, without an external reference solution or a stronger teacher. By further removing invalid-action turns, ASCENT distills enhanced privileged experience for more efficient execution.
The authors characterize ASCENT's population target and the limits of sparse outcome selection. Across ALFWorld, WebShop, and AppWorld at varied model scales, ASCENT improves task success and interaction efficiency as experience accumulates, outperforms online adaptation methods, and transfers to held-out scenes. According to the paper, exact success rises by more than 22 points over the base on ALFWorld and WebShop at both model scales, with fewer turns per episode.
The work suggests that an agent can consolidate verified experience into its weights without a separate training phase or memory retrieval. A project page is available at artificer-ai-lab.github.io/ASCENT.