B1710
Title: More efficient exact permutation testing: using a representative subgroup
Authors: Nick Koning - Erasmus University Rotterdam (Netherlands) [presenting]
Jesse Hemerik - Leiden University Medical Centre (Netherlands)
Abstract: Non-parametric tests based on permutation, rotation or sign-flipping are examples of so-called group-invariance tests. These tests test the invariance of the null distribution under a set of transformations that has a group structure, in the algebraic sense. Such groups are often huge, which makes it computationally infeasible to test using the entire group. Hence, it is standard practice to test using a randomly sampled set of transformations from the group. This random sample still needs to be substantial to obtain good power and replicability. We improve upon the standard practice by using a well-designed subgroup of transformations instead of a random sample. The resulting subgroup-invariance test is still exact, as invariance under a group implies invariance under its subgroups. We illustrate this in a generalized location model and find that it can yield a more powerful and fully replicable test with the same number of transformations. For the special case of a normal location model and a particular design of the subgroup, we show that the power improvement is equivalent to the power difference between a Monte Carlo Z-test and a Monte Carlo t-test. In our simulations, we find that our test has the same power as a test based on sampling that uses twice as many random transformations.
