GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework

• DOI: 10.1609/aaai.v38i14.29472
• Publication Date: 2024-03-25T11:31:43Z
• Authors: Fangyikang Wang; Huminhao Zhu; Chao Zhang; Hanbin Zhao; Hui Qian
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Bayesian sampling; Computer Science - Artificial Intelligence; Science; Physics; QC1-999; Q; Astrophysics; 01 natural sciences; Article; Machine Learning (cs.LG); QB460-466; Artificial Intelligence (cs.AI); 0101 mathematics; probability gradient flow; variational inference

Particle-based Variational Inference (ParVI) methods approximate the target distribution by iteratively evolving finite weighted particle systems. Recent advances of ParVI methods reveal the benefits of accelerated position update strategies and dynamic weight adjustment approaches. In this paper, we propose the first ParVI framework that possesses both accelerated position update and dynamical weight adjustment simultaneously, named the General Accelerated Dynamic-Weight Particle-based Variational Inference (GAD-PVI) framework. Generally, GAD-PVI simulates the semi-Hamiltonian gradient flow on a novel Information-Fisher-Rao space, which yields an additional decrease on the local functional dissipation. GAD-PVI is compatible with different dissimilarity functionals and associated smoothing approaches under three information metrics. Experiments on both synthetic and real-world data demonstrate the faster convergence and reduced approximation error of GAD-PVI methods over the state-of-the-art.