A0228
Title: Splitting Langevin dynamics for Bayesian neural networks
Authors: Neil Chada - City University of Hong Kong (Hong Kong) [presenting]
Abstract: A scalable kinetic Langevin dynamics algorithm is proposed for sampling parameter spaces of big data and AI applications. The scheme combines a symmetric forward/backward sweep over minibatches with a symmetric discretization of Langevin dynamics. For a particular Langevin splitting method (UBU), it is shown that the resulting symmetric minibatch splitting-UBU (SMS-UBU) integrator has bias O(h2d1/2) in dimension d>0 with stepsize h>0, despite only using one minibatch per iteration, thus providing excellent control of the sampling bias as a function of the stepsize. The algorithm is applied to explore local modes of the posterior distribution of Bayesian neural networks (BNNs) and evaluate the calibration performance of the posterior predictive probabilities for neural networks with convolutional neural network architectures for classification problems on three different datasets (Fashion-MNIST, Celeb-A, and chest X-ray). Results indicate that BNNs sampled with SMS-UBU can offer significantly better calibration performance compared to standard methods of training and stochastic weight averaging
