B0751
Title: A new variational family for Bayesian deep learning
Authors: Susan Wei - University of Melbourne (Australia) [presenting]
Abstract: Unlike in regular statistical models, the posterior distribution over neural network weights is not asymptotically Gaussian. As established in singular learning theory, the posterior distribution over the parameters of a singular model is, asymptotically, a mixture of standard forms. Loosely, this means the parameter space can be partitioned such that in each local parameter set, the average log-likelihood ratio can be made normal crossing via an algebraic-geometrical transform known as a resolution map. We leverage this under-appreciated result to propose a new variational family for Bayesian deep learning. Affine coupling layers are employed to learn the unknown resolution map, effectively rendering the proposed methodology a normalizing flow with the generalized gamma as the source distribution, rather than the multivariate Gaussian typically employed.
