Title: Evaluating predictive bias in the presence of differential label noise
Authors: Alexandra Chouldechova - Carnegie Mellon University (United States) [presenting]
Abstract: Risk assessment tools are widely used around the country to inform decision making within the criminal justice system. Recently, considerable attention has been devoted to the question of whether such tools may suffer from racial bias. In this type of assessment, a fundamental issue is that the training and evaluation of the model is based on a variable (arrest) that may represent a noisy version of an unobserved outcome of more central interest (offense). We present a sensitivity analysis framework for assessing how differential label noise affects the predictive bias properties of the risk assessment model as a predictor of reoffense. We also discuss the impact of differential label noise on the model training process.