A0307
Title: How source, topic, \& recency influence LLM-based sentiment predictions: Evidence from cross-sectional return prediction
Authors: Jule Schuettler - University of St.Gallen (Switzerland) [presenting]
Abstract: The purpose is to investigate the predictive value of sentiment extracted from diverse financial text sources using state-of-the-art large language models (LLMs). In addition, it is examined whether incorporating topic information can enhance sentiment classification. The analysis is based on a comprehensive dataset comprising earnings call transcripts, newspaper headlines, and social media tweets. Using fine-tuned DeBERTa models, long/short portfolios are constructed based on sentiment predictions, and their trading performance is evaluated across sources. It is found that, while all sources contain predictive information, only tweets and headlines yield profitable strategies once trading costs are considered. Furthermore, it is demonstrated that combining multiple sources improves predictive performance by reducing portfolio volatility. A novel topic-aware LLM that incorporates topic information through unsupervised topic modeling is also introduced, and trading strategies based on this model are shown to outperform those that do not account for topical context. Finally, it is found that a rolling window training approach significantly enhances model performance, underscoring the importance of recency in dynamic financial environments.
