B0627
Title: Rapture of the deep: Highs and lows of Bayes in a world of depths
Authors: Julyan Arbel - Inria (France) [presenting]
Abstract: Bayesian deep learning is appealing as it combines the coherence and natural uncertainty quantification of the Bayesian paradigm with the expressivity and compositional flexibility of deep neural networks. Besides, it has the potential to provide learning mechanisms endowed with certain interpretability guarantees. An overview of the distributional properties of Bayesian neural networks is made. This journey starts with an early 90s study. This led to the so-called Gaussian hypothesis of the pre-activations, which can be justified when the number of neurons per layer tends to infinity. This hypothesis is then contrasted with recent work on heavy-tailed pre-activations. Finally, a set of constraints is described that a neural network should fulfil to ensure Gaussian pre-activations.
