Existing evaluations of agentic AI systems tend to focus on task success, which offers only limited insight into how agents behave when the evidence they retrieve contradicts what their underlying model already "believes." A new paper posted to Hugging Face, titled "Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict," proposes a different lens: epistemic humility.
The authors define epistemic humility (EH) as an agent's willingness to recognize uncertainty, act on it, and communicate it during task execution. They operationalize the concept through three trajectory-level behavioral dimensions — Identify, Solve, and Escalate (ISE) — and study it under conditions of knowledge conflict, where a backbone model's parametric knowledge clashes with encountered evidence, or where two contextual sources disagree.
The evaluation covers two conflict settings: a controlled conflict and a naturally occurring conflict during multi-step agentic execution, each paired with matched no-conflict controls. Four agents were tested.
The central finding is that higher task accuracy does not necessarily correspond to greater epistemic humility. Some high-accuracy configurations detected conflicts during execution but still failed to communicate unresolved uncertainty in their incorrect final answers. Trajectory-level analysis showed agents often spot conflicts in early steps but fail to maintain or resolve them later.
The paper also reports that model-level interventions can improve epistemic humility, though often at the cost of task accuracy — suggesting that humility emerges from the interaction among the backbone model, the agent harness, and the evaluation environment rather than from any single component.