A1012
Title: Economic uncertainty measures, experts and ChatGPT
Authors: Stanislaw Bartha - Ernst and Young (Poland)
Maria Elena Bontempi - University of Bologna (Italy)
Svetlana Makarova - University College London (United Kingdom) [presenting]
Abstract: The aims of the paper are (1) to propose and apply a method of evaluating the quality of macroeconomic uncertainty measures and (2) to apply this method for comparing the accuracy of predicting increases in uncertainty by four such measures constructed for Poland. The method uses a simple statistical bootstrap-based test to verify the hypothesis that measures' jumps generated by uncertainty are purely random. That is, it compares the frequency of the cases where the increases in uncertainty showed by the tested measure coincide with the date of an uncertainty-generating event identified by experts without prior knowledge of the measure. The results show that for Poland, uncertainty measures based on Google Trends searches are superior to those based on forecast disagreement, but only if the search criteria are well-defined. Access to a high-quality expert panel might be difficult in practice, so we have repeated the test using the ChatGPT events identification instead of the experts. The ChatGPT results were slightly less reliable in testing than the experts-based
