A0296
Title: Narrative shift detection: A hybrid approach of dynamic topic models and large language models
Authors: Kai-Robin Lange - TU Dortmund University (Germany) [presenting]
Abstract: As narratives in media evolve rapidly, understanding and investigating how narratives develop over time has become increasingly important. While large language models (LLMs) are effective at capturing narrative elements, small research groups might struggle to apply them across entire corpora due to high computational and financial costs. To address this issue, a method is introduced that combines the language understanding capabilities of LLMs with the scalability of dynamic topic models to analyze narrative shifts over time, utilizing the narrative policy framework. A dynamic topic model, along with a change point detection algorithm, is used to identify topical changes. Documents representative of these changes are then selected and analyzed using an LLM to automatically interpret the nature of the change and distinguish between narrative shifts and mere content shifts. This approach is applied to a corpus of "The Wall Street Journal" articles spanning 2009 to 2023. Results suggest that LLMs are effective at extracting narrative shifts when such shifts are present, but are limited when distinguishing between content changes and genuine narrative transitions.
