A0660
Title: Optimal combinations of mean square error and directional forecast accuracy for model selection
Authors: Robert Kunst - Institute for Advanced Studies (Austria) [presenting]
Mauro Costantini - Sapienza University of Rome (Italy)
Abstract: Forecasting economic time series often aims at two targets that may be in conflict: Accuracy in the sense of closeness between forecast and realized value, as it is measured by mean square error, and directional forecast error, with interest exclusively focused on ups and downs. In this context, the problem of selecting a forecast model is studied among two assumed candidates. Minimizing a combined loss function that accounts for both targets jointly is considered, using a weighting scheme. In previous work, Monte Carlo analysis under different scenarios has explored the strength of the procedure. Time-homogeneous univariate and vector autoregressions have been considered, but also generating laws that involve thresholds and structural breaks. Some windows have been identified where the weighted combined targets succeed in improving the accuracy criteria. The interaction is investigated between the weight assigned to directional accuracy in the selection criterion and in the evaluation criterion. One may conjecture that a strong weight for directional accuracy (DA) implies a stronger DA performance of the selected model, but this is not necessarily the case. Simulation evidence of realistic designs is presented, where a stronger DA weight implies deteriorating DA performance. In some cases, an optimal weight can be found where DA performance attains a maximum. In empirical applications, such simulations may be helpful in optimizing the DA weight in the criterion function.
