A1009
Title: Analysis on dynamics of deep neural network with high-dimensional structure
Authors: Masaaki Imaizumi - The University of Tokyo (Japan) [presenting]
Abstract: Several topics related to the connection between statistics, machine learning, and dynamical systems are introduced. The first topic concerns the learning of the linearly non-separable structure by a neural network with simultaneous training. The result shows that a two-layer neural network can learn a linearly non-separable function even when both layers are updated simultaneously. To establish this result, the fine-grained tracking of neuron variability is characterized. The second topic discusses precise dynamical analysis of a deep neural network with a realistic architecture. Although the existing precise dynamical analysis depends on the infinite-width limit, the analysis allows finite-width and resolves some its limitations. The state evolution approach is applied to describe Gaussian fluctuations in the first-layer values and deterministic transform in the subsequent layers.
