B1256
Title: Time-uniform conformal and probably approximately correct prediction
Authors: Kayla Scharfstein - Carnegie Mellon University (United States) [presenting]
Arun Kuchibhotla - Carnegie Mellon University (United States)
Abstract: Given that machine learning algorithms are increasingly being deployed to aid in high-stakes decision-making, uncertainty quantification methods that wrap around these black box models, such as conformal prediction, have received much attention in recent years. Unfortunately, conformal prediction will not produce valid prediction intervals if the size of the dataset of interest is not known in advance, which is often the case in sequential settings. As such, an extension of the conformal prediction and related probably approximately correct (PAC) prediction frameworks to sequential settings are developed where the number of data points is not fixed in advance. The resulting prediction sets are anytime-valid since they can be constructed at any time chosen by the analyst, even if this choice depends on the data. Theoretical guarantees are presented for the proposed methods, and their validity and utility on simulated and real datasets are demonstrated.
