Researchers have published a paper on Hugging Face arguing that existing confidence estimators for language models share a limiting assumption: they judge confidence only from the current inference process, whether by introspection, token-probability scoring, or resampling. The authors contend that the current inference alone is not a sufficient basis for confidence.
Their proposal, XConf (eXperiential Confidence), estimates confidence alongside the model's accumulated experience. That experience is stored as a record of the model's own graded past episodes, each containing the task, the model's reflection, its stated confidence, the outcome, and a lesson written once the grade arrived.
XConf works in two stages. A Recall stage retrieves past episodes involving similar tasks met with a similar stated confidence and reads off their historical success rate. A Reflect stage then shows the model this record, has it name its recurring failure mode, and restate a confidence now informed by its own track record.
The authors describe the estimator as format-general, requiring no logit access or weight updates and costing only one answer generation. Across nine benchmarks spanning reasoning, coding, multimodal QA, and interactive agents, and four models from three families, they report that XConf beats or matches ten-sample self-consistency in discrimination (AUROC) on 23 of 24 comparisons, with much lower calibration error (ECE) at a tenth of the generation cost.
For selective prediction, the paper states that abstaining on the 10% least-confident episodes raises the delivered success rate by up to 8.7 points on agent tasks. The authors position experiential confidence estimation as a new paradigm for future general-purpose confidence estimation.