A0181
Title: Exploring the weighted Fisher divergence for high-dimensional Gaussian variational approximation
Authors: Aoxiang Chen - National University of Singapore (Singapore)
David Nott - National University of Singapore (Singapore)
Siew Li Linda Tan - National University of Singapore (Singapore) [presenting]
Abstract: The purpose is to explore the use of the weighted Fisher divergence, which focuses on gradient differences between the target and variational densities and includes the Fisher and score-based divergences as special cases in Gaussian variational inference. Its behavior is studied under the mean-field assumption for both Gaussian and non-Gaussian targets and is used to develop inference for high-dimensional hierarchical models where posterior conditional independence structure is captured via a sparse precision matrix. Using stochastic gradient descent to enforce sparsity, two approaches are developed for minimizing the weighted Fisher divergence based separately on the reparametrization trick and a minibatch approximation of the objective. A truncation trick is also proposed to reduce underestimation in uncertainty. Applications include generalized linear mixed models and state space models.
