EcoSta 2017: Start Registration
View Submission - EcoSta2017
A0389
Title: Statistical inference for model parameters with stochastic gradient descent Authors:  Xi Chen - New York University (United States) [presenting]
Abstract: The problem of statistical inference of the true model parameters is investigated based on stochastic gradient descent (SGD). To this end, we propose two consistent estimators of the asymptotic covariance of the average iterate from SGD: (1) an intuitive plug-in estimator and (2) a computationally more efficient batch-means estimator, which only uses the iterates from SGD. As the SGD process forms a time-inhomogeneous Markov chain, our batch-means estimator with carefully chosen increasing batch sizes generalizes the classical batch-means estimator designed for time-homogenous Markov chains. Both proposed estimators allow us to construct asymptotically exact confidence intervals and hypothesis tests. We further discuss an extension to conducting inference based on SGD for high-dimensional linear regression.