A0372
Title: Online estimation and inference for robust policy evaluation in reinforcement learning
Authors: Yichen Zhang - Purdue University (United States) [presenting]
Abstract: Reinforcement learning has emerged as one of the prominent topics attracting attention in modern statistical learning, with policy evaluation being a key component. Unlike the traditional machine learning literature on this topic, statistical inference is emphasized for the model parameters and value functions of reinforcement learning algorithms. While most existing analyses assume random rewards to follow standard distributions, the concept of robust statistics is embraced in reinforcement learning by simultaneously addressing issues of outlier contamination and heavy-tailed rewards within a unified framework. A fully online robust policy evaluation procedure is developed, and the Bahadur-type representation of the estimator is established. Furthermore, an online procedure is developed to efficiently conduct statistical inference based on the asymptotic distribution. Robust statistics and statistical inference in reinforcement learning are connected, offering a more versatile and reliable approach to online policy evaluation. Finally, the efficacy of the algorithm is validated through numerical experiments conducted in simulations and real-world reinforcement learning experiments.
