A1473
Title: Comparing calibration fairness across ensemble learning methods: Bagging, boosting, and stacking
Authors: Hiroe Seto - Kyoto Women\'s University (Japan) [presenting]
Michio Yamamoto - The University of Osaka / RIKEN AIP / Shiga University (Japan)
Hikari Komai - Kyoto Women-s University (Japan)
Kuniyoshi Hayashi - Kyoto Women-s University (Japan)
Abstract: Fairness in prediction models has become a critical ethical concern in society. In particular, attention has focused on calibration fairness that is, whether predicted probabilities are equally accurate across subgroups defined by sensitive attributes such as gender and race. However, methods for evaluating calibration fairness have not been sufficiently established. To address this issue, several approaches for this purpose have been recently proposed, including the calibration error difference (CED) and the variable-based calibration plot (VC-plot). In this context, the question of which prediction algorithms can achieve fairer calibration remains unresolved. Identifying algorithms that are prone to achieving fair calibration is a key step toward constructing fair probabilistic prediction models. Previous work has suggested that ensemble learning methods may yield fairer calibration than non-ensemble approaches. The investigation focuses on which types of ensemble learning methods can achieve calibration fairness in probabilistic prediction models. Specifically, calibration fairness is systematically compared among a single decision tree (DT) and three ensemble learning methods bagging (random forest; RF), boosting (gradient boosting decision tree; GBDT), and stacking (DT+GBDT, DT+GBDT+RF). Using three real-world datasets requiring fairness consideration, probabilistic prediction models are constructed and evaluated with CED and VC-plot via 100 bootstrap iterations.
