Online Posterior Sampling with a Diffusion Prior

• DOI: 10.48550/arxiv.2410.03919
• Publication Date: 2024-10-04
• Authors: Branislav Kveton; Boris N. Oreshkin; Youngsuk Park; Aniket Anand Desh...; Rui Song
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Statistics - Machine Learning; Machine Learning (stat.ML); Machine Learning (cs.LG)

Posterior sampling in contextual bandits with a Gaussian prior can be implemented exactly or approximately using the Laplace approximation. The is computationally efficient but it cannot describe complex distributions. In this work, we propose approximate posterior algorithms for diffusion model prior. key idea to sample from chain of conditional posteriors, one each stage reverse process, which are estimated closed form Our approximations motivated by prior, and inherit its simplicity efficiency. They asymptotically consistent perform well empirically on variety bandit problems.