A0998
Title: Estimating and implementing conventional fairness metrics with probabilistic protected features
Authors: Hadi Elzayn - Meta (United States)
Emily Black - Stanford University (United States)
Patrick Vossler - Stanford University (United States) [presenting]
Nathan Jo - Stanford University (United States)
Abstract: Techniques from algorithmic fairness are increasingly used to train models that satisfy various quantitative notions of fairness, hoping to avoid potential bad outcomes observed in various machine learning-based settings. Yet the vast majority of such techniques require access to the protected attribute, either at train time or in production, and this protected attribute is often unavailable. It is shown how to leverage probabilistic race imputation, most notably via Bayesian Improved Surname Geocoding (BISG), along with special estimators, to estimate quantitative fairness measurements without access to the protected feature. Under certain conditions described, these estimators will be unbiased; otherwise, they can be used as bounds. This technique can be used both simply to document or rule out unfairness (according to the particular definition) when given a model. It can be incorporated into many techniques designed to produce fair models. Quantitative guarantees are proved for the proposed methods, and two empirical illustrations are provided based on tax data from the IRS and medical data from AFC.
