A0949
Title: A nonparametric statistics approach to variable selection in deep neural networks with theoretical guarantees
Authors: Long Feng - University of Hong Kong (Hong Kong) [presenting]
Abstract: The purpose is to address the problem of variable selection in an extremely challenging setting: $\mathbb{E}(y \mid \mathbf{x}) = G(\mathbf{x}_{\mathcal{S}_0})$, where $\mathcal{S}_0$ is the target variable set, and $G$ is an arbitrary unknown non-linear function with certain basic smoothness properties, such as being H{\"o}lder smooth. To achieve this goal, variable selection is first explored in deep neural networks. The strong approximation capability of neural networks is then leveraged, and the proposed approach for any H{\"o}lder smooth function is generalized. Unlike typical optimization-based deep learning methods, neural networks are formulated into index models, and it is proposed to estimate $\mathcal{S}_0$ using the second-order Stein's formula. The approach is not only computationally efficient by avoiding the gradient-descent-type algorithm for solving highly nonconvex deep-learning-related optimizations, but more importantly, it can theoretically guarantee variable selection consistency for deep neural networks and arbitrary H{\"o}lder smooth functions when the sample size $n = \Omega(p^2)$, where $p$ is the dimension of the input. Comprehensive simulations and real genetic data analyses further demonstrate the superior performance of the approach.
