Model-free posterior inference on the area under the receiver operating characteristic curve

• DOI: 10.1016/j.jspi.2020.03.008
• Publication Date: 2020-04-10T00:35:30Z
• Authors: Zhe Wang; Ryan Martin
• Keywords: Methodology (stat.ME); FOS: Computer and information sciences; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; 0101 mathematics; 01 natural sciences; Statistics - Methodology

The area under the receiver operating characteristic curve (AUC) serves as a summary of a binary classifier's performance. Methods for estimating the AUC have been developed under a binormality assumption which restricts the distribution of the score produced by the classifier. However, this assumption introduces an infinite-dimensional nuisance parameter and can be inappropriate, especially in the context of machine learning. This motivates us to adopt a model-free Gibbs posterior distribution for the AUC. We present the asymptotic Gibbs posterior concentration rate, and a strategy for tuning the learning rate so that the corresponding credible intervals achieve the nominal frequentist coverage probability. Simulation experiments and a real data analysis demonstrate the Gibbs posterior's strong performance compared to existing methods based on a rank likelihood.<br/>19 pages, 3 figures, 5 tables<br/>