Model Calibration

Calibrating Large Language Models with Sample Consistency

Qing Lyu, Kumar Shridhar, Chaitanya Malaviya et al.

Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing

Jaroslaw Blasiok, Preetum Nakkiran

Reasoning Models Better Express Their Confidence

Dongkeun Yoon, Seungone Kim, Sohee Yang et al.

Addressing Misspecification in Simulation-based Inference through Data-driven Calibration

Antoine Wehenkel, Juan L. Gamella, Ozan Sener et al.

Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models

Gianni Franchi, Olivier Laurent, Maxence Leguéry et al.

ConfTuner: Training Large Language Models to Express Their Confidence Verbally

Yibo Li, Miao Xiong, Jiaying Wu et al.

On Temperature Scaling and Conformal Prediction of Deep Classifiers

Lahav Dabah, Tom Tirer

Simultaneous Swap Regret Minimization via KL-Calibration

Haipeng Luo, Spandan Senapati, Vatsal Sharan

Epistemic Uncertainty Quantification For Pre-Trained Neural Networks

Hanjing Wang, Qiang Ji

Feature Clipping for Uncertainty Calibration

Linwei Tao, Minjing Dong, Chang Xu

QA-Calibration of Language Model Confidence Scores

Putra Manggala, Atalanti A Mastakouri, Elke Kirschbaum et al.

AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration

Javier Tirado-Garín, Javier Civera

Generalized Venn and Venn-Abers Calibration with Applications in Conformal Prediction

Lars van der Laan, Ahmed Alaa

Unlocking the Potential of Model Calibration in Federated Learning

Yun-Wei Chu, Dong-Jun Han, Seyyedali Hosseinalipour et al.

Towards Calibrated Deep Clustering Network

Yuheng Jia, Jianhong Cheng, Hui LIU et al.

Calibrating LLMs with Information-Theoretic Evidential Deep Learning

Yawei Li, David Rügamer, Bernd Bischl et al.

Discretization-free Multicalibration through Loss Minimization over Tree Ensembles

Hongyi Henry Jin, Zijun Ding, Dung Daniel Ngo et al.

Calibrated Language Models and How to Find Them with Label Smoothing

Jerry Huang, Peng Lu, QIUHAO Zeng

Beyond One-Hot Labels: Semantic Mixing for Model Calibration

Haoyang Luo, Linwei Tao, Minjing Dong et al.

Leveraging Uncertainty Estimates To Improve Classifier Performance

Gundeep Arora, Srujana Merugu, Anoop Saladi et al.

Model Uncertainty Quantification by Conformal Prediction in Continual Learning

Rui Gao, Weiwei Liu

Combining Priors with Experience: Confidence Calibration Based on Binomial Process Modeling

Jinzong Dong, Zhaohui Jiang, Dong Pan et al.

Real-Time Calibration Model for Low-Cost Sensor in Fine-Grained Time Series

Seokho Ahn, Hyungjin Kim, Sungbok Shin et al.

Modeling Stereo-Confidence out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep

Jae Young Lee, Woonghyun Ka, Jaehyun Choi et al.

Towards Adversarial Robustness via Debiased High-Confidence Logit Alignment

Kejia Zhang, Juanjuan Weng, Zhiming Luo et al.

Multivariate Latent Recalibration for Conditional Normalizing Flows

Victor Dheur, Souhaib Ben Taieb

Quantifying Uncertainty in the Presence of Distribution Shifts

Yuli Slavutsky, David Blei

Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator

Beier Luo, Shuoyuan Wang, Sharon Li et al.

Conformal Prediction Beyond the Horizon: Distribution-Free Inference for Policy Evaluation

Feichen Gan, Lu Youcun, Yingying Zhang et al.

Taming Overconfidence in LLMs: Reward Calibration in RLHF

Jixuan Leng, Chengsong Huang, Banghua Zhu et al.

Calibrating Expressions of Certainty

Peiqi Wang, Barbara Lam, Yingcheng Liu et al.

Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity

Victor Li, Baiting Chen, Yuzhen Mao et al.

Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibration

Thomas Decker, Volker Tresp, Florian Buettner

SteerConf: Steering LLMs for Confidence Elicitation

Ziang Zhou, Tianyuan Jin, Jieming Shi et al.

On Calibration of LLM-based Guard Models for Reliable Content Moderation

Hongfu Liu, Hengguan Huang, Xiangming Gu et al.

Conformal Linguistic Calibration: Trading-off between Factuality and Specificity

Zhengping Jiang, Anqi Liu, Ben Van Durme

Understanding Model Calibration - A gentle introduction and visual exploration of calibration and the expected calibration error (ECE)

Maja Pavlovic

Know What You Don't Know: Uncertainty Calibration of Process Reward Models

Young-Jin Park, Kristjan Greenewald, Kaveh Alimohammadi et al.

On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines

Selim Kuzucu, Kemal Oksuz, Jonathan Sadeghi et al.

Provable Uncertainty Decomposition via Higher-Order Calibration

Gustaf Ahdritz, Aravind Gollakota, Parikshit Gopalan et al.

Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs

Gerardo Flores, Alyssa H. Smith, Julia Fukuyama et al.

Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers

Thomas Klein, Sascha Meyen, Wieland Brendel et al.

Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence

Dharmesh Tailor, Alvaro Correia, Eric Nalisnick et al.

Towards Unbiased Calibration using Meta-Regularization

Jacek Golebiowski, Cheng Wang

Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling

Yufan Li, Pragya Sur

General Uncertainty Estimation with Delta Variances

Simon Schmitt, John Shawe-Taylor, Hado van Hasselt

Reassessing How to Compare and Improve the Calibration of Machine Learning Models

Muthu Chidambaram, Rong Ge

Dirichlet-Based Prediction Calibration for Learning with Noisy Labels

Chen-Chen Zong, Ye-Wen Wang, Ming-Kun Xie et al.

Catalyst for Clustering-Based Unsupervised Object Re-identification: Feature Calibration

Huafeng Li, Qingsong Hu, Zhanxuan Hu

On the Asymptotic Optimality of Confidence Interval Based Algorithms for Fixed Confidence MABs

Kushal Kejriwal, Nikhil Karamchandani, Jayakrishnan Nair

Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space

Mohsin Hasan, Guojun Zhang, Kaiyang Guo et al.

Intelligent Calibration for Bias Reduction in Sentiment Corpora Annotation Process

Idan Toker, David Sarne, Jonathan Schler

Towards Certification of Uncertainty Calibration under Adversarial Attacks

Cornelius Emde, Francesco Pinto, Thomas Lukasiewicz et al.

Generative Calibration of Inaccurate Annotation for Label Distribution Learning

Liang He, Yunan Lu, Weiwei Li et al.

Attack-inspired Calibration Loss for Calibrating Crack Recognition

Zhuangzhuang Chen, Qiangyu Chen, Jiahao Zhang et al.

CLIB-FIQA: Face Image Quality Assessment with Confidence Calibration

Fu-Zhao Ou, Chongyi Li, Shiqi Wang et al.

Improving Model Probability Calibration by Integration of Large Data Sources with Biased Labels

Renat Sergazinov, Richard Chen, Cheng Ji et al.

Conformalized Interval Arithmetic with Symmetric Calibration

Rui Luo, Zhixin Zhou

Parametric ρ-Norm Scaling Calibration

Siyuan Zhang, Linbo Xie

Inlier Confidence Calibration for Point Cloud Registration

Yongzhe Yuan, Yue Wu, Xiaolong Fan et al.

Self-Calibrating Vicinal Risk Minimisation for Model Calibration

Jiawei Liu, Changkun Ye, Ruikai Cui et al.

Calibration Bottleneck: Over-compressed Representations are Less Calibratable

Deng-Bao Wang, Min-Ling Zhang

Parametric Scaling Law of Tuning Bias in Conformal Prediction

Hao Zeng, Kangdao Liu, Bingyi Jing et al.

How Flawed Is ECE? An Analysis via Logit Smoothing

Muthu Chidambaram, Holden Lee, Colin McSwiggen et al.

Rectifying Conformity Scores for Better Conditional Coverage

Vincent Plassier, Alexander Fishkov, Victor Dheur et al.

Set Learning for Accurate and Calibrated Models

Lukas Muttenthaler, Robert A Vandermeulen, Qiuyi (Richard) Zhang et al.

Linguistic Calibration of Long-Form Generations

Neil Band, Xuechen Li, Tengyu Ma et al.

Pointwise Information Measures as Confidence Estimators in Deep Neural Networks: A Comparative Study

Shelvia Wongso, Rohan Ghosh, Mehul Motani

Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models

Shuoyuan Wang, Sharon Li, Hongxin Wei

Improving Multi-Class Calibration through Normalization-Aware Isotonic Techniques

Alon Arad, Saharon Rosset

Minimal Perspective Autocalibration

Andrea Porfiri Dal Cin, Timothy Duff, Luca Magri et al.

Unbiased Estimator for Distorted Conics in Camera Calibration

Chaehyeon Song, Jaeho Shin, Myung-Hwan Jeon et al.

PAC-Bayes Analysis for Recalibration in Classification

Masahiro Fujisawa, Futoshi Futami

T-Cal: An Optimal Test for the Calibration of Predictive Models

Donghwan Lee, Xinmeng Huang, Hamed Hassani et al.

Learning model uncertainty as variance-minimizing instance weights

Nishant Jain, Karthikeyan Shanmugam, Pradeep Shenoy

Improving the Statistical Efficiency of Cross-Conformal Prediction

Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs

Miao Xiong, Zhiyuan Hu, Xinyang Lu et al.

Uncertainty Quantification for LLM-Based Survey Simulations

Chengpiao Huang, Yuhang Wu, Kaizheng Wang

Multicalibration for Confidence Scoring in LLMs

Gianluca Detommaso, Martin A Bertran, Riccardo Fogliato et al.

Enhancing Post-training Quantization Calibration through Contrastive Learning

Yuzhang Shang, Gaowen Liu, Ramana Kompella et al.

Algorithms with Calibrated Machine Learning Predictions

Judy Hanwen Shen, Ellen Vitercik, Anders Wikum

Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs

Daniel D. Johnson, Daniel Tarlow, David Duvenaud et al.

LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses

Xin Liu, Muhammad Khalifa, Lu Wang

FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler

Hongyi Peng, Han Yu, Xiaoli Tang et al.

An Empirical Study Into What Matters for Calibrating Vision-Language Models

Weijie Tu, Weijian Deng, Dylan Campbell et al.

On the Calibration of Human Pose Estimation

Kerui Gu, Rongyu Chen, Xuanlong Yu et al.

Open-Vocabulary Calibration for Fine-tuned CLIP

Shuoyuan Wang, Jindong Wang, Guoqing Wang et al.

Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration

Gyusang Cho, Chan-Hyun Youn

Sampling-based Multi-dimensional Recalibration

Youngseog Chung, Ian Char, Jeff Schneider

Conformalized Survival Distributions: A Generic Post-Process to Increase Calibration

Shi-ang Qi, Yakun Yu, Russell Greiner

Confidence Self-Calibration for Multi-Label Class-Incremental Learning

Kaile Du, Yifan Zhou, Fan Lyu et al.

Uncertainty Calibration with Energy Based Instance-wise Scaling in the Wild Dataset

Mijoo Kim, Junseok Kwon

Instant Uncertainty Calibration of NeRFs Using a Meta-Calibrator

Niki Amini-Naieni, Tomas Jakab, Andrea Vedaldi et al.

Adaptive Bounding Box Uncertainties via Two-Step Conformal Prediction

Alexander Timans, Christoph-Nikolas Straehle, Kaspar Sakmann et al.

IMG: Calibrating Diffusion Models via Implicit Multimodal Guidance

Jiayi Guo, Chuanhao Yan, Xingqian Xu et al.

Improving Accuracy and Calibration via Differentiated Deep Mutual Learning

Han Liu, Peng Cui, Bingning Wang et al.

Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles

Eric Slyman, Mehrab Tanjim, Kushal Kafle et al.

CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning

Jinsoo Bae, Seoung Bum Kim, Hyungrok Do

Uncertainty Weighted Gradients for Model Calibration

Jinxu Lin, Linwei Tao, Minjing Dong et al.

Deterministic Object Pose Confidence Region Estimation

Jinghao Wang, Zhang Li, Zi Wang et al.