A fast universal self-tuned sampler within Gibbs sampling

• DOI: 10.1016/j.dsp.2015.04.005
• Publication Date: 2015-04-17T11:02:22Z
• Authors: L. Martino; H. Yang; D. Luengo; J. Kanniainen; J. Corander
• Keywords: Markov Chain Monte Carlo (MCMC); Gibbs sampling; Bayesian inference; 0202 electrical engineering, electronic engineering, information engineering; Metropolis within Gibbs; 02 engineering and technology; Adaptive rejection Metropolis sampling

Bayesian inference often requires efficient numerical approximation algorithms, such as sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) methods. The Gibbs sampler is a well-known MCMC technique, widely applied in many signal processing problems. Drawing samples from univariate full-conditional distributions efficiently is essential for the practical application of the Gibbs sampler. In this work, we present a simple, self-tuned and extremely efficient MCMC algorithm which produces virtually independent samples from these univariate target densities. The proposal density used is self-tuned and tailored to the specific target, but it is not adaptive. Instead, the proposal is adjusted during an initial optimization stage, following a simple and extremely effective procedure. Hence, we have named the newly proposed approach as FUSS (Fast Universal Self-tuned Sampler), as it can be used to sample from any bounded univariate distribution and also from any bounded multi-variate distribution, either directly or by embedding it within a Gibbs sampler. Numerical experiments, on several synthetic data sets (including a challenging parameter estimation problem in a chaotic system) and a high-dimensional financial signal processing problem, show its good performance in terms of speed and estimation accuracy.