Title: Scoring functions for forecasts of multivariate distributions and level sets
Authors: Xiaochun Meng - University of Sussex (United Kingdom) [presenting]
James Taylor - University of Oxford (United Kingdom)
Souhaib Ben Taieb - University of Mons (Belgium)
Siran Li - Rice University (United States)
Abstract: Interest in the prediction of multivariate probability distributions is growing due to the increasing availability of rich datasets and computational developments. Scoring functions enable the comparison of forecast accuracy, and can potentially be used for estimation. A scoring function for multivariate distributions that has gained some popularity is the energy score. This is a generalization of the continuous ranked probability score (CRPS), which is widely used for univariate distributions. A little-known, alternative generalization is the multivariate CRPS (MCRPS). We propose a new theoretical framework for scoring functions for multivariate distributions, which encompasses the energy score and multivariate CRPS as specific cases. This framework can be used to generate new scores, and we demonstrate this with the introduction of a score based on the density function. For univariate distributions, it is well-established that the CRPS can be expressed as the integral over the quantile check loss score. We show that, in a similar way, scoring functions for multivariate distributions can be disintegrated to obtain the scoring functions for three types of level sets: projection quantiles, isoprobabilty contours and density contours. To compute the various scoring functions, we propose a simple numerical algorithm. We use a simulation study to support our findings.