A1498
Title: Statistical reproducibility for Tukey's honest significant difference test
Authors: Norah Alshahrani - University of Bisha (Saudi Arabia) [presenting]
Abstract: Reproducibility is a cornerstone of reliable research and has received increasing attention in recent years. Statistical reproducibility, in particular, concerns whether statistical conclusions remain consistent when experiments are replicated under identical conditions. This paper investigates the reproducibility probability (RP) of Tukey's Honest Significant Difference (HSD) test, a widely used post hoc procedure following one-way ANOVA, within the framework of nonparametric predictive inference (NPI). Simulation results show that RP declines as p-values approach the significance threshold, indicating instability in such cases. Moreover, sample size exerts a notable influence: larger samples decrease RP when p-value > alpha, but increase RP when the p-value < alpha. Distributional characteristics also affect RP, especially when groups have unequal variances. These findings highlight the importance of jointly considering p-values, sample size, and distributional features when interpreting the reproducibility of post hoc test results.
