A new research paper from Hugging Face, titled LLMs Get Lost in Evolving User Intent, highlights a fundamental flaw in current large language models: their inability to track and act on user intent as it changes over the course of a conversation. The authors argue that while LLMs are increasingly used as collaborative agents in dynamic interactions, they are still predominantly evaluated on static, single-turn tasks where user intent is fully specified upfront.
To bridge this gap, the researchers introduce a framework that transforms existing static benchmarks into dynamic multi-turn conversations. In these conversations, user intent is incrementally revealed, revised, or redirected across turns, while preserving the original evaluation protocol. This allows for controlled testing without requiring new annotations.
Across multiple tasks, the study finds a consistent phenomenon: strong performance in static settings does not transfer to the evolving-intent setting. All tested model families showed substantial drops in performance, indicating that LLMs fail to faithfully track shifting user goals.
The paper resonates with real-world observations from practitioners. One developer noted that agents often treat the first user request as final, leading to what they call 'intent drift'—where the agent either locks onto the initial version or over-corrects to the latest utterance, forgetting earlier constraints. Another commenter highlighted that models can become 'locally consistent while globally misaligned' with the original objective, suggesting a need for explicit intent-tracking mechanisms.
The authors conclude that the ability to handle evolving intent is critical for future collaborative agents, yet it remains invisible to current static evaluations. They hope the framework will become a standard for evaluating LLMs in realistic, dynamic interactions.