Validating Bayesian Inference Algorithms with Simulation-Based Calibration

• DOI: 10.48550/arxiv.1804.06788
• Publication Date: 2018-01-01
• Authors: Talts, Sean; Betancourt, Michael; Simpson, Daniel; Vehtari, Aki; Gelman, Andrew
• Keywords: Methodology (stat.ME); FOS: Computer and information sciences; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; 0101 mathematics; 01 natural sciences; Statistics - Methodology

Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce \emph{simulation-based calibration} (SBC), a general procedure for validating inferences from Bayesian algorithms capable of generating posterior samples. This procedure not only identifies inaccurate computation and inconsistencies in model implementations but also provides graphical summaries that can indicate the nature of the problems that arise. We argue that SBC is a critical part of a robust Bayesian workflow, as well as being a useful tool for those developing computational algorithms and statistical software.<br/>19 pages, 13 figures<br/>