A1001
Title: Deep learning at smaller scale
Authors: Rebekka Burkholz - CISPA Helmholtz Center for Information Security (Germany) [presenting]
Abstract: Deep learning continues to achieve impressive breakthroughs across disciplines but relies on increasingly large neural network models that are trained on massive data sets. Their development inflicts costs that are only affordable by a few labs and prevents global participation in the creation of related technologies. The focus is on the question of whether it really has to be like this, and some of the major challenges that limit the success of deep learning on smaller scales are discussed. Three examples of complimentary approaches are given that could help addressing the underlying issues: (i) early neural network sparsification, (ii) the integration of useful inductive bias in the design of problem specific neural network architectures, and (iii) improving training from scratch.
