A1609
Title: A two-phase A/B test for binary outcomes on network data
Authors: Nicholas Rios - George Mason University (United States) [presenting]
Abstract: A/B testing is a powerful and popular tool for conducting online experiments for comparing a treatment with a control. Many applications for A/B testing involve comparing binary outcomes at the user level, such as monitoring whether or not a user clicked a button. Additionally, many of these individual users belong to a network, such as users of Facebook and LinkedIn. Unfortunately, when individuals belong to a network, the Stable Unit Treatment Value Assumption (SUTVA) can be violated because connected individuals tend to influence each other's decisions and responses. Additionally, in practice, both the mechanism of network interference and the true model for the response are unknown. An online sampling framework is proposed that dynamically minimizes a design optimality criterion based on the worst-case treatment variance under a set of plausible models and network interference mechanisms. Simulations and numerical studies on synthetic data based on real networks from Facebook users show that the proposed framework is competitive with existing approaches in terms of power and Type-I error rates.
