A0640
Title: PUMP: Estimating power when adjusting for multiple outcomes in multi-level experiments
Authors: Kristen Hunter - UNSW (Australia) [presenting]
Kristin Porter - KE Porter Consulting LLC (United States)
Luke Miratrix - Harvard University (United States)
Abstract: For randomized controlled trials (RCTs) with a single intervention's impact being measured on multiple outcomes, researchers often apply a multiple testing procedure (MTP) (such as Bonferroni or Benjamini-Hochberg) to adjust p-values. Such an adjustment reduces the likelihood of spurious findings but also changes the statistical power, sometimes substantially, which reduces the probability of detecting effects when they do exist. However, this consideration is frequently ignored in typical power analyses, as existing tools do not easily accommodate the use of MTPs. The PUMP R package is introduced as a tool for analysts to estimate statistical power, minimum detectable effect size (MDES), and sample size requirements for multi-level RCTs with multiple outcomes. One of PUMP's main innovations is accommodating multiple outcomes: power estimates from PUMP properly account for the adjustment in p-values from applying an MTP. Also, PUMP allows researchers to consider a variety of definitions of power in order to choose the most appropriate types of power for the goals of their study. The package supports a variety of commonly-used frequentist multi-level RCT designs and linear mixed effects models. In addition to the main functionality of estimating power, MDES, and sample size requirements, the package allows the user to easily explore the sensitivity of these quantities to changes in underlying assumptions.
