A0362
Title: Advances in Bayesian neural model selection
Authors: Alexander Immer - ETH Zurich (Switzerland) [presenting]
Abstract: Choosing optimal hyperparameters for deep learning can be highly expensive due to trial-and-error procedures and required expertise. Conceptually, a Bayesian approach to hyperparameter selection could help overcome such issues because it can rely on gradient-based optimization and does not require a held-out validation set. However, such an approach requires estimation and differentiation of the marginal likelihood, which is inherently intractable. Recent advances in Laplace approximations are discussed, which provide efficient estimates and enable optimizing hyperparameters with stochastic gradients just like neural network weights. Further, successful applications of Bayesian model selection are demonstrated, and shortcomings of current algorithms are discussed.
