A1091
Title: Natural gradient variational Bayes without Fisher matrix analytic calculation and its inversion
Authors: Antoine Godichon-Baggioni - Sorbonne-Universite (France)
Duy Nguyen - Marist College (United States)
Minh-Ngoc Tran - University of Sydney (Australia)
Minh-Ngoc Tran - University of Sydney (Australia) [presenting]
Abstract: The purpose is to introduce a method for efficiently approximating the inverse of the Fisher information matrix, a crucial step in achieving effective variational Bayes inference. A notable aspect of the approach is the avoidance of analytically computing the Fisher information matrix and its explicit inversion. Instead, an iterative procedure is introduced to generate a sequence of matrices that converge to the inverse of Fisher information. The natural gradient variational Bayes algorithm without analytic expression of the Fisher matrix and its inversion is provably convergent. It achieves a convergence rate of order O(logs/s), with s the number of iterations. A central limit theorem for the iterates is also obtained. The implementation of the method does not require the storage of large matrices, and it achieves a linear complexity in the number of variational parameters. The algorithm exhibits versatility, making it applicable across a diverse array of variational Bayes domains, including Gaussian approximation and normalizing flow Variational Bayes. A range of numerical examples is offered to demonstrate the efficiency and reliability of the proposed variational Bayes method.
