A0263
Title: Dropout regularization versus L2-penalization in the linear model
Authors: Johannes Schmidt-Hieber - University of Twente (Netherlands) [presenting]
Sophie Langer - University Twente (Germany)
Gabriel Clara - University of Twente (Netherlands)
Abstract: The focus is on the statistical behavior of gradient descent iterates with dropout in the linear regression model. In particular, non-asymptotic bounds for expectations and covariance matrices of the iterates are derived. In contrast with the widely cited connection between dropout and L2-regularization in expectation, the results indicate a much more subtle relationship, owing to interactions between the gradient descent dynamics and the additional randomness induced by dropout. We also study a simplified variant of dropout that does not have a regularizing effect and converges with the least squares estimator.
