A0879
Title: Robust Bayesian A/B testing
Authors: Boris Choy - University of Sydney (Australia) [presenting]
Xuan Li - University of New South Wales (Austria)
Abstract: Bayesian A/B testing has been widely adopted in industry practice. Most practitioners typically focus on large-scale inference from A/B testing and assume collected data follows a Gaussian sampling distribution by applying the central limit theorem (CLT). However, not every company can afford to have a large amount of data for analysis. In this case, the normality assumption held by CLT may not be applicable. To address this issue, the aim is to relax the normality assumption and provide robust Bayesian A/B testing models that can be used for both large and small datasets. By applying the heavy-tailed distributions under the Bayesian hierarchical framework, the proposed models stand out from other existing models on both large- and small-scale A/B testing.
