A0949
Title: Doubly robust interval estimation for optimal policy evaluation in online learning
Authors: Hengrui Cai - University of California Irvine (United States) [presenting]
Abstract: Evaluating the performance of an ongoing policy plays a vital role in many areas, such as medicine and economics, to provide crucial instruction on the early stop of the online experiment and timely feedback from the environment. Policy evaluation in online learning thus attracts increasing attention by inferring the mean outcome of the optimal policy (i.e., the value) in real time. Yet, such a problem is particularly challenging due to the dependent data generated in the online environment, the unknown optimal policy, and the complex exploration and exploitation trade-off in the adaptive experiment. The aim is to overcome these difficulties in policy evaluation for online learning. The probability of exploration that quantifies the probability of exploring the non-optimal actions is explicitly derived under commonly used bandit algorithms. This probability of conducting valid inference is used on the online conditional mean estimator under each action, and the doubly robust interval estimation (DREAM) method is developed to infer the value under the estimated optimal policy in online learning. The proposed value estimator provides double protection on the consistency and is asymptotically normal with a Wald-type confidence interval provided. Extensive simulations and real data applications are conducted to demonstrate the empirical validity of the proposed DREAM method.
