A1103
Title: On the number of replications in resampling tests and Monte Carlo simulation studies
Authors: Daniel Gaigall - FH Aachen University of Applied Sciences (Germany) [presenting]
Julian Gerstenberg - Goethe University Frankfurt (Germany)
Abstract: The purpose is to investigate rejection probabilities of statistical tests based on resampling procedures. The general framework under consideration covers, in particular, bootstrap and permutation techniques. It turns out that specific properties of the P-value distribution play a key role, namely convexity or concavity, the Bernstein property, and those of beta mixture models. A detailed analysis is provided, and it is clarified how these properties relate to each other. New bounds are derived for the rejection probability. The results link the number of replications with size and power of the test. Numerical considerations demonstrate the quality of the bounds. An important application is the nested simulation estimator in Monte Carlo simulation studies. Findings indicate that a moderate or even rather small number of replications is sufficient to obtain useful simulation results. This enables a substantial reduction of the computational effort in Monte Carlo simulation studies.
