A1196
Title: Numerical validation of a doubly randomized significance test for difference-in-differences estimation
Authors: Stanislaw Halkiewicz - AGH University of Cracow (Poland) [presenting]
Andrzej Kaluza - AGH University of Cracow (Poland)
Abstract: The focus is on the numerical validation of a simulation-based significance test for the difference-in-differences (DiD) estimator, previously developed and accepted for publication. The method relies on independent randomization of both treatment and time indicators to generate a reference distribution for the DiD significance under the null hypothesis. Its statistical properties are assessed through extensive numerical experiments. Simulations are conducted across a wide range of outcome distributions, including normal, skewed, and heavy-tailed, as well as varying sample sizes. The results demonstrate that the test consistently yields symmetric and stable rejection intervals, converges rapidly, and remains invariant under linear transformations of the metric. Compared to conventional OLS-based approaches, the test shows improved control of type I error, particularly in small or imbalanced samples. The effect of sample size is also examined on the spread of the test distribution, and a strong negative correlation is confirmed with its interquartile range. Finally, the method is applied to real-world data from Indonesia's INPRES program to illustrate practical implications. All experiments are fully reproducible using the accompanying Python code.
