A1683
Title: Small-sample behavior of the test for comparing standardized mean differences in meta-analysis
Authors: Anya Umlauf - University of California San Diego (United States) [presenting]
Abstract: Two common estimators for the difference between standardized means are Cohen's $d$ and Hedges' $g$. For either estimate, a comparison of two effect sizes can be done with a test based on normal distribution assumption. Much has been written on test performance for large sample sizes (more than 10 per group). Group sizes below ten are not unusual, however, particularly in pilot studies or studies in areas with limited resources. Having a reliable test for such a situation would be desirable. The test for comparing two effect sizes is dependent on the variances for the effect sizes being compared. There are several ways of estimating the effect size variance for both Cohen's $d$ and Hedges' $g$. We set out to investigate which method leads to the most accurate results when group sizes are small. We simulated tests under normality assumption using gamma-based estimates, as well as the large-sample approximation and unbiased estimate, both proposed by Hedges. We also tested a pooled estimate calculated from the unbiased variance estimates. We expected the approximated statistics to yield biased results for small samples, but the simulations showed the large-sample estimate outperformed other approximations. Tests based on Hedges' g were more accurate than the tests for Cohen's d.
