B1744
Title: How robust is the skill score of probabilistic earthquake forecasts?
Authors: Alessia Caponera - University of Milano-Bicocca (Italy) [presenting]
Maximilian Werner - University of Bristol and London Mathematical Laboratory (United Kingdom)
Abstract: The reproducibility of results across experiments is a fundamental pillar of science. A number of recent studies have failed to reproduce, however, the results of several landmark studies in psychology, biomedicine, economics, and geophysics. A factor contributing to irreproducibility may be the non-ergodicity of some systems: when the time-average does not equal the ensemble average, classical statistical inference procedures face challenges. We investigate the reproducibility of the predictive skill of probabilistic models that forecast the spatio-temporal evolution of earthquake sequences. Using time-dependent forecasts created during the 2010-2012 M7.2 Darfield (New Zealand) earthquake sequence, we compare the variability of models time-averaged predictive skill with commonly assumed ensemble confidence bounds. To understand the variability of predictive skill across multiple earthquake sequences, we simulate long earthquake catalogues from a widely-used earthquake clustering model (a Hawkes point process) and assess the convergence of the time-averaged predictive skill to the ensemble average. Our results suggest that many earthquake sequences are required before the temporal average converges to the ensemble average. Commonly used uncertainty estimates of the predictive skill are too optimistic, and therefore apparently significant differences between model performance may not indicate future performance.
