Model Calibration
Calibrating confidence estimates
Related Topics (Robustness)
Top Papers
Conformal Risk Control
Anastasios Angelopoulos, Stephen Bates, Adam Fisch et al.
Calibrating Large Language Models with Sample Consistency
Qing Lyu, Kumar Shridhar, Chaitanya Malaviya et al.
Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
Jaroslaw Blasiok, Preetum Nakkiran
Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement
Jaehun Jung, Faeze Brahman, Yejin Choi
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
Rylan Schaeffer, Hailey Schoelkopf, Brando Miranda et al.
Reasoning Models Better Express Their Confidence
Dongkeun Yoon, Seungone Kim, Sohee Yang et al.
Copula Conformal prediction for multi-step time series prediction
Sophia Sun, Rose Yu
GeoCalib: Learning Single-image Calibration with Geometric Optimization
Alexander Veicht, Paul-Edouard Sarlin, Philipp Lindenberger et al.
Addressing Misspecification in Simulation-based Inference through Data-driven Calibration
Antoine Wehenkel, Juan L. Gamella, Ozan Sener et al.
A Call to Reflect on Evaluation Practices for Age Estimation: Comparative Analysis of the State-of-the-Art and a Unified Benchmark
Jakub Paplham, Vojtech Franc
Bounds on Representation-Induced Confounding Bias for Treatment Effect Estimation
Valentyn Melnychuk, Dennis Frauen, Stefan Feuerriegel
A Unified Comparative Study with Generalized Conformity Scores for Multi-Output Conformal Regression
Victor Dheur, Matteo Fontana, Yorick Estievenart et al.
Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumption
Du CHEN, Tianhe Wu, Kede Ma et al.
Kandinsky Conformal Prediction: Efficient Calibration of Image Segmentation Algorithms
Joren Brunekreef, Eric Marcus, Ray Sheombarsing et al.
Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models
Gianni Franchi, Olivier Laurent, Maxence Leguéry et al.
PAC Prediction Sets Under Label Shift
Wenwen Si, Sangdon Park, Insup Lee et al.
R-EDL: Relaxing Nonessential Settings of Evidential Deep Learning
Mengyuan Chen, Junyu Gao, Changsheng Xu
Conformal Thresholded Intervals for Efficient Regression
Rui Luo, Zhixin Zhou
Confidence Estimation for Error Detection in Text-to-SQL Systems
Oleg Somov, Elena Tutubalina
Consistency Checks for Language Model Forecasters
Daniel Paleka, Abhimanyu Pallavi Sudhir, Alejandro Alvarez et al.
Reliable and Efficient Amortized Model-based Evaluation
Sang Truong, Yuheng Tu, Percy Liang et al.
Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction
Wei Qian, Chenxu Zhao, Yangyi Li et al.
ConfTuner: Training Large Language Models to Express Their Confidence Verbally
Yibo Li, Miao Xiong, Jiaying Wu et al.
On Temperature Scaling and Conformal Prediction of Deep Classifiers
Lahav Dabah, Tom Tirer
Foundation Model-oriented Robustness: Robust Image Model Evaluation with Pretrained Models
Peiyan Zhang, Haoyang Liu, Chaozhuo Li et al.
Unraveling Batch Normalization for Realistic Test-Time Adaptation
Zixian Su, Jingwei Guo, Kai Yao et al.
Noise Calibration and Spatial-Frequency Interactive Network for STEM Image Enhancement
Hesong Li, Ziqi Wu, Ruiwen Shao et al.
Error-quantified Conformal Inference for Time Series
Junxi Wu, Dongjian Hu, Yajie Bao et al.
On the Limitations of Temperature Scaling for Distributions with Overlaps
Muthu Chidambaram, Rong Ge
Conformal Linguistic Calibration: Trading-off between Factuality and Specificity
Zhengping Jiang, Anqi Liu, Ben Van Durme
Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks
Ziqing Wang, Yuetong Fang, Jiahang Cao et al.
Error Bounds for Gaussian Process Regression Under Bounded Support Noise with Applications to Safety Certification
Robert Reed, Luca Laurenti, Morteza Lahijanian
Robustness Auditing for Linear Regression: To Singularity and Beyond
Ittai Rubinstein, Samuel Hopkins
SteerConf: Steering LLMs for Confidence Elicitation
Ziang Zhou, Tianyuan Jin, Jieming Shi et al.
Epistemic Uncertainty Quantification For Pre-Trained Neural Networks
Hanjing Wang, Qiang Ji
The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothing
Blaise Delattre, Alexandre Araujo, Quentin Barthélemy et al.
CSformer: Combining Channel Independence and Mixing for Robust Multivariate Time Series Forecasting
Haoxin Wang, Yipeng Mo, Kunlan Xiang et al.
Simultaneous Swap Regret Minimization via KL-Calibration
Haipeng Luo, Spandan Senapati, Vatsal Sharan
Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination’s Impact on Machine Translation
Muhammed Yusuf Kocyigit, Eleftheria Briakou, Daniel Deutsch et al.
Calibrating Expressions of Certainty
Peiqi Wang, Barbara Lam, Yingcheng Liu et al.
A Generic Framework for Conformal Fairness
Aditya Vadlamani, Anutam Srinivasan, Pranav Maneriker et al.
On Volume Minimization in Conformal Regression
Batiste Le Bars, Pierre Humbert
Feature Clipping for Uncertainty Calibration
Linwei Tao, Minjing Dong, Chang Xu
Integral Imprecise Probability Metrics
Siu Lun (Alan) Chau, Michele Caprio, Krikamol Muandet
Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting
Wei Chen, Yuxuan Liang
Constructing Confidence Intervals for Average Treatment Effects from Multiple Datasets
Yuxin Wang, Maresa Schröder, Dennis Frauen et al.
Revisiting Calibration of Wide-Angle Radially Symmetric Cameras
Andrea Porfiri Dal Cin, Francesco Azzoni, Giacomo Boracchi et al.
QA-Calibration of Language Model Confidence Scores
Putra Manggala, Atalanti A Mastakouri, Elke Kirschbaum et al.
Difficulty-aware Balancing Margin Loss for Long-tailed Recognition
Minseok Son, Inyong Koo, Jinyoung Park et al.
AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration
Javier Tirado-Garín, Javier Civera
Robust Self-calibration of Focal Lengths from the Fundamental Matrix
Viktor Kocur, Daniel Kyselica, Zuzana Kukelova
Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inference
Dongyan Huo, Yudong Chen, Qiaomin Xie
Conformal Inference of Individual Treatment Effects Using Conditional Density Estimates
Baozhen Wang, Xingye Qiao
Generalized Venn and Venn-Abers Calibration with Applications in Conformal Prediction
Lars van der Laan, Ahmed Alaa
Kernel-based Optimally Weighted Conformal Time-Series Prediction
Jonghyeok Lee, Chen Xu, Yao Xie
Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
David Heineman, Valentin Hofmann, Ian Magnusson et al.
Quantifying Prediction Consistency Under Fine-tuning Multiplicity in Tabular LLMs
Faisal Hamman, Sachindra P Dissanayake, Saumitra Mishra et al.
Conformal Prediction for Ensembles: Improving Efficiency via Score-Based Aggregation
Yash Patel, Eduardo Ochoa Rivera, Ambuj Tewari
Introducing FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark
Zhangdie Yuan, Zifeng Ding, Andreas Vlachos
Backward Conformal Prediction
Etienne Gauthier, Francis Bach, Michael Jordan
$\texttt{BetaConform}$: Efficient MAP Estimation of LLM Ensemble Judgment Performance with Prior Transfer
Huaizhi Qu, Inyoung Choi, Zhen Tan et al.
Non-parametric Sensor Noise Modeling and Synthesis
Ali Mosleh, Luxi Zhao, Atin Vikram Singh et al.
Towards Establishing Guaranteed Error for Learned Database Operations
Sepanta Zeighami, Cyrus Shahabi
Unlocking the Potential of Model Calibration in Federated Learning
Yun-Wei Chu, Dong-Jun Han, Seyyedali Hosseinalipour et al.
Simplification Is All You Need against Out-of-Distribution Overconfidence
Keke Tang, Chao Hou, Weilong Peng et al.
Towards Robust Influence Functions with Flat Validation Minima
Xichen Ye, Yifan Wu, Weizhong Zhang et al.
Multi-Accurate CATE is Robust to Unknown Covariate Shifts
Angela Zhou, Christoph Kern, Michael Kim
Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification
Kaizheng Wang, Fabio Cuzzolin, Keivan Shariatmadar et al.
How Benchmark Prediction from Fewer Data Misses the Mark
Guanhua Zhang, Florian E. Dorner, Moritz Hardt
Uncertainty Weighted Gradients for Model Calibration
Jinxu Lin, Linwei Tao, Minjing Dong et al.
Towards Calibrated Deep Clustering Network
Yuheng Jia, Jianhong Cheng, Hui LIU et al.
Calibrating LLMs with Information-Theoretic Evidential Deep Learning
Yawei Li, David Rügamer, Bernd Bischl et al.
High-Dimensional Calibration from Swap Regret
Maxwell Fishelson, Noah Golowich, Mehryar Mohri et al.
Probably Approximately Precision and Recall Learning
Lee Cohen, Yishay Mansour, Shay Moran et al.
Human-in-the-Loop Visual Re-ID for Population Size Estimation
Gustavo Perez, Daniel Sheldon, Grant Van Horn et al.
Multi-Dimensional Conformal Prediction
Yam Tawachi, Bracha Laufer-Goldshtein
Fractal Calibration for Long-tailed Object Detection
Konstantinos Alexandridis, Ismail Elezi, Jiankang Deng et al.
Consistency-Guided Temperature Scaling Using Style and Content Information for Out-of-Domain Calibration
Wonjeong Choi, Jungwuk Park, Dong-Jun Han et al.
Discretization-free Multicalibration through Loss Minimization over Tree Ensembles
Hongyi Henry Jin, Zijun Ding, Dung Daniel Ngo et al.
From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers
Swaminathan Gurumurthy, Karnik Ram, Bingqing Chen et al.
Learning With Multi-Group Guarantees For Clusterable Subpopulations
Jessica Dai, Nika Haghtalab, Eric Zhao
FreeCap: Hybrid Calibration-Free Motion Capture in Open Environments
Aoru Xue, Yiming Ren, Zining Song et al.
Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning
Yan Scholten, Stephan Günnemann
How Much is Unseen Depends Chiefly on Information About the Seen
Seongmin Lee, Marcel Boehme
Rethinking Classifier Re-Training in Long-Tailed Recognition: Label Over-Smooth Can Balance
Siyu Sun, Han Lu, Jiangtong Li et al.
CBMA: Improving Conformal Prediction through Bayesian Model Averaging
Pankaj Bhagwat, Linglong Kong, Bei Jiang
Stochastic Online Conformal Prediction with Semi-Bandit Feedback
Haosen Ge, Hamsa Bastani, Osbert Bastani
Exact Recovery of Sparse Binary Vectors from Generalized Linear Measurements
Arya Mazumdar, Neha Sangwan
Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization
Sunay Joshi, Shayan Kiyani, George J. Pappas et al.
Learning multivariate Gaussians with imperfect advice
Arnab Bhattacharyya, Davin Choo, Philips George John et al.
Beyond One-Hot Labels: Semantic Mixing for Model Calibration
Haoyang Luo, Linwei Tao, Minjing Dong et al.
Credal Prediction based on Relative Likelihood
Timo Löhr, Paul Hofman, Felix Mohr et al.
Towards Certification of Uncertainty Calibration under Adversarial Attacks
Cornelius Emde, Francesco Pinto, Thomas Lukasiewicz et al.
Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models
Sima Noorani, Shayan Kiyani, George J. Pappas et al.
Calibrated Language Models and How to Find Them with Label Smoothing
Jerry Huang, Peng Lu, QIUHAO Zeng
MC-PanDA: Mask Confidence for Panoptic Domain Adaptation
Ivan Martinovic, Josip Šarić, Siniša Šegvić
T-CIL: Temperature Scaling using Adversarial Perturbation for Calibration in Class-Incremental Learning
Seong-Hyeon Hwang, Minsu Kim, Steven Euijong Whang
RC-AutoCalib: An End-to-End Radar-Camera Automatic Calibration Network
Van-Tin Luu, Yong-Lin Cai, Vu-Hoang Tran et al.
Uncertainty-Aware Self-Training for CTC-Based Automatic Speech Recognition
Eungbeom Kim, Kyogu Lee
Pushing the Limits of BFP on Narrow Precision LLM Inference
Hui Wang, Yuan Cheng, Xiaomeng Han et al.