Large language models (LLMs) are typically evaluated in a static, zero-shot manner, ignoring their ability to improve through interaction. A new research paper from Hugging Face introduces Chain-of-Experience (CoE), a framework that enables LLMs to learn from iterative experience at test time, forming a continual improvement loop.
CoE works by accumulating experiential traces through iterative interactions with self or environmental feedback. The study instantiates CoE with diverse feedback mechanisms, including model self-feedback and environmental signals such as correctness or public coding test pass rates. The approach was evaluated across math, coding, and knowledge domains using 8 LLMs, including GPT-5, Gemini-2.5 Pro, and Claude-4.5 Sonnet.
Results show that leveraging iterative experience consistently outperforms feedback-free baselines. With self-feedback alone, CoE achieves a 5.6% overall improvement and 19% lower API cost across tasks and models. Combining complementary feedback channels (e.g., model and correctness signals) yields additional gains, and CoE delivers higher accuracy per token than existing test-time strategies.
The study also finds a positive correlation between LLM base ability and improvement capacity, and shows that models remain robust under weak or spurious feedback. Different feedback types contribute to distinct improvement aspects, with most gains emerging early in the iterations.