A1321
Title: Matrix-exponential variational inference
Authors: Martin Magris - ITAM - Instituto Tecnologico Autonomo de Mexico (Mexico) [presenting]
Abstract: Variational Inference (VI) has become an important tool for approximating complex posterior distributions in modern Bayesian applications, particularly in machine learning, where models are often non-conjugate and high-dimensional. A widely used approach is Gaussian fixed-form VI. While this choice enables scalable optimization with gradient-based methods, enforcing the positive definiteness of the covariance matrix $\Sigma$ remains challenging. A novel and analytically tractable parameterization of the variational covariance matrix is proposed. Specifically, the variational covariance matrix $\Sigma$ is represented as $\Sigma = MDM^{\top}$, where $M = \exp(\Omega)$ is the matrix exponential of a skew-symmetric matrix and $D$ is a diagonal matrix. This construction guarantees positive-definiteness while preserving numerical stability and computational efficiency. Under this parameterization, closed-form expressions for both the standard and natural gradients of the Evidence Lower Bound are derived and a close connection to Riemann optimization is established. Efficient algorithms for computing derivatives of the matrix exponential are developed and the method is demonstrated on logistic regression, time-series models with GARCH volatility, and Gaussian mixture models. The proposed approach yields stable posterior approximations comparable to those of existing VI and MCMC methods while enabling fast and scalable optimization.
