A0828
Title: Model multiplicity in policy learning
Authors: Hannah Busshoff - University of St. Gallen (Switzerland) [presenting]
Abstract: In policy learning, treatment rules are learned from data that map individual characteristics to treatment recommendations, aiding decision-making in fields like healthcare, education, and public policy. While existing literature derives asymptotic properties of policy learners, it lacks methods to assess the reliability of these rules learned from finite data. Policy rules are derived by choosing the best treatment rule according to training data without accounting for model multiplicity. Model multiplicity refers to the condition where multiple models have similar empirical performance but may imply different individual-level consequences. In deployment, this can cause treatment decisions to be unstable, arbitrary, and unjustified. Formal measures are introduced to quantify the extent of model multiplicity in policy learning. Extending integer programming tools, we analyze decision-relevant model multiplicity, identifying individuals who receive conflicting treatment recommendations from empirically plausible policy rules. The method is demonstrated with synthetic data and applied to two real-world scenarios: allocating individuals to training programs and prescribing tranexamic acid to trauma patients. The applications illustrate the behavior of model multiplicity in two high-stake domains, highlighting its implications for policy reliability.
