We advocate an optimization-centric view on and introduce a novel generalization of Bayesian inference. Our inspiration is the representation Bayes' rule as infinite-dimensional optimization problem (Csiszar, 1975; Donsker Varadhan; 1975, Zellner; 1988). First, we use it to prove optimality result standard Variational Inference (VI): Under proposed view, Evidence Lower Bound (ELBO) maximizing VI posterior preferable alternative approximations posterior. Next, argue for generalizing The need...

When it is acknowledged that all candidate parameterised statistical models are misspecified relative to the data generating process, decision maker (DM) must currently concern themselves with inference for parameter value minimising Kullback–Leibler (KL)-divergence between model and this process (Walker, 2013). However, has long been known KL-divergence places a large weight on correctly capturing tails of sample distribution. As result, DM required worry about robustness their tail...

Abstract Statisticians often face the choice between using probability models or a paradigm defined by minimising loss function. Both approaches are useful and, if can be re-cast into proper model, there many tools to decide which model is more appropriate for observed data, in sense of explaining data's nature. However, when leads an improper no principled ways guide this choice. We address task combining Hyvärinen score, naturally targets infinitesimal relative probabilities, and general...

Probabilistic programming methods have revolutionised Bayesian inference, making it easier than ever for practitioners to perform Markov-chain-Monte-Carlo sampling from non-conjugate posterior distributions. Here we focus on Stan, arguably the most used probabilistic tool inference (Carpenter et al., 2017), and its interface with R via brms (Burkner, 2017) rstanarm (Goodrich 2024) packages. Although easy implement, these tools can become computationally prohibitive when applied datasets many...

We propose a Bayesian hidden Markov model for analyzing time series and sequential data where special structure of the transition probability matrix is embedded to explicit-duration semi-Markovian dynamics. Our formulation allows development highly flexible interpretable models that can integrate available prior information on state durations while keeping moderate computational cost perform efficient posterior inference. show benefits choosing approach HSMM estimation over its frequentist...

This note is a collection of several discussions the paper "Beyond subjective and objective in statistics", read by A. Gelman C. Hennig to Royal Statistical Society on April 12, 2017, appear Journal Society, Series

Increasing interest in privacy-preserving machine learning has led to new and evolved approaches for generating private synthetic data from undisclosed real data. However, mechanisms of privacy preservation can significantly reduce the utility data, which turn impacts downstream tasks such as predictive models or inference. We propose several re-weighting strategies using privatised likelihood ratios that not only mitigate statistical bias estimators but also have general applicability...

ABSTRACT A frequent challenge when using graphical models in practice is that the sample size limited relative to number of parameters. They also become hard interpret variables p gets large. We consider applications where one has external data, form networks between variables, can improve inference and help fitted model. An example interest regards interplay social media co-evolution COVID-19 pandemic across USA counties. develop a spike-and-slab prior framework depicts how partial...

We present the very first robust Bayesian Online Changepoint Detection algorithm through General Inference (GBI) with $β$-divergences. The resulting inference procedure is doubly for both parameter and changepoint (CP) posterior, linear time constant space complexity. provide a construction exponential models demonstrate it on Linear Regression model. In so doing, we make two additional contributions: Firstly, GBI scalable using Structural Variational approximations that are exact as $β\to...

We study the stability of posterior predictive inferences to specification likelihood model and perturbations data generating process. In modern big analyses, decision-maker may elicit useful broad structural judgements but a level interpolation is required arrive at model. One model, often computationally convenient canonical form, chosen, when many alternatives would have been equally consistent with elicited judgements. Equally, observational datasets contain unforeseen heterogeneities...

We propose a sparse vector autoregressive (VAR) hidden semi-Markov model (HSMM) for modeling temporal and contemporaneous (e.g. spatial) dependencies in multivariate nonstationary time series. The HSMM's generic state distribution is embedded special transition matrix structure, facilitating efficient likelihood evaluations arbitrary approximation accuracy. To promote sparsity of the VAR coefficients, we deploy an $l_1$-ball projection prior, which combines differentiability with positive...

Differential privacy guarantees allow the results of a statistical analysis involving sensitive data to be released without compromising any individual taking part. Achieving such generally requires injection noise, either directly into parameter estimates or estimation process. Instead artificially introducing perturbations, sampling from Bayesian posterior distributions has been shown special case exponential mechanism, producing consistent, and efficient private altering generative The...

There is significant growth and interest in the use of synthetic data as an enabler for machine learning environments where release real restricted due to privacy or availability constraints. Despite a large number methods generation, there are comparatively few results on statistical properties models learnt data, fewer still situations researcher wishes augment with another party's synthesised data. We Bayesian paradigm characterise updating model parameters when these settings,...

We propose a Bayesian hidden Markov model for analyzing time series and sequential data where special structure of the transition probability matrix is embedded to explicit-duration semi-Markovian dynamics. Our formulation allows development highly flexible interpretable models that can integrate available prior information on state durations while keeping moderate computational cost perform efficient posterior inference. show benefits choosing approach HSMM estimation over its frequentist...

We consider two applications where we study how dependence structure between many variables is linked to external network data. first the interplay social media connectedness and co-evolution of COVID-19 pandemic across USA counties. next stock market returns firms relates similarities in economic policy indicators from text regulatory filings. Both are modelled via Gaussian graphical models one has develop spike-and-slab LASSO frameworks integrate data, both facilitating interpretation...