CMStatistics 2022: Start Registration
View Submission - CMStatistics
B0693
Title: A data adaptive rank-based procedure for assessing reproducibility of high-throughput experiments Authors:  Wen Zhou - Colorado State University (United States) [presenting]
Debashis Ghosh - University of Colorado Anschutz Medical Campus (United States)
Austin Ellingworth - Colorado State University (United States)
Abstract: Reproducibility guarantees the consistency and validity of experimental findings, while the lack of which can lead to negative and even catastrophic effects on scientific discovery. In high-throughput studies, reproducibility has often been identified as hypotheses with coinciding test results across different experiments. The maximum rank statistic (MaRR) was introduced to identify reproducible hypotheses based on the agreement of test results across experiments. Regardless of its empirical success, the theoretical guarantees of MaRR remain largely unknown. We carefully investigate MaRR which lends it to quantifying reproducibility in high-throughput studies. We also develop a novel data adaptive rank-based statistic that balances the signal strength of a hypothesis and its variation across experiments. Based on the new statistic, we design a procedure to assess reproducibility with marginal false discovery rate (mFDR) control. By inspecting the rejection region, we show that the new procedure dominates the original MaRR statistic with superior power. We also present a revealing phase transition phenomenon of our procedure using the bivariate Gaussian canonical model. Using comprehensive simulations, we demonstrate the finite sample performance of our method, which corroborates the theoretical findings.