Post-hoc calibration is widely used to correct the confidence scores of machine learning models, but it can inadvertently alter the model's top-1 prediction—the class with the highest probability. While accuracy metrics capture the net effect on correctness, they do not reveal how often predictions actually change. To address this, researchers have proposed a new method called Calibrator-Output Repair for Top-1 Decision Preservation (CORD).
CORD is a post-fit adapter that works by repairing the full calibrated probability vector. It uses only the original and calibrated outputs to determine the mass assigned to the original top-1 class, then redistributes the remaining probability mass over the other classes according to the calibrated conditional distribution. This ensures that the repaired vector's argmax recovers the original prediction, achieving a zero Top-1 Prediction Change Rate (TPCR) by construction.
Importantly, CORD does not alter the fitted calibrator or its direct output, does not train any additional supervised model, and requires no user-tuned hyperparameters. On the calibration split, it coordinates the repaired masses to retain the calibrated outputs' mean mass on original predictions whenever possible.
Experiments on CIFAR-10, CIFAR-100, and ImageNet-1K show that CORD not only achieves zero TPCR but also lowers mean Expected Calibration Error (ECE), Negative Log-Likelihood (NLL), and Brier score compared to the direct calibrated outputs in every dataset. These improvements persist under distribution shift and across different calibration-set sizes.
The method effectively removes the prediction-preservation constraint from calibrator fitting, allowing calibrators to focus on confidence accuracy while CORD handles decision preservation. The code is available on GitHub, and the paper is on arXiv.