A0225
Title: Forecast calibration, backtests, and score decompositions for Value-at-Risk
Authors: Marius Puke - University of Hohenheim (Germany) [presenting]
Timo Dimitriadis - Heidelberg University (Germany)
Abstract: The evaluation of Value-at-Risk (VaR) forecasts is fundamental to the stability of our financial system through the regulatory frameworks Basel III for banks and Solvency II for insurance, but also their internal risk forecasts. While the statistical literature advises the use of scoring (loss) functions for forecast evaluation, the economic literature, as well as the practical implementation in the regulatory systems, is still mainly concerned with so-called VaR backtests. The fundamental connection between these two principles is still not well understood, which is especially problematic as these approaches regularly deliver contrasting results in practice. We formally connect these concepts by drawing on the recent literature on forecast calibration. For this, we make use of recently developed decompositions of scoring functions into interpretable components assessing (mis)calibration, discrimination, and uncertainty and show that the backtests only assess calibration while entirely ignoring the forecasts discriminating ability. Intuitively, this corresponds to ignoring the forecast's ability to discriminate between periods of lower and higher risk. As a consequence, we propose the practical use of score decompositions that, in addition to backtests, reveal information on the overall predictive ability and the hitherto unexploited information on discrimination and hence provoke more informative insights in applications.
