A1602
Title: Dynamic decision-making under model misspecification: A stochastic stability approach
Authors: Xinyu Dai - Brown University (United States) [presenting]
Daniel Chen - Brown University (United States)
Yian Qian - Brown University (United States)
Abstract: Dynamic decision-making under model uncertainty is central to many economic environments, yet existing bandit and reinforcement learning algorithms rely on the assumption of correct model specification. The behavior and performance of Thompson Sampling (TS), one of the most commonly used Bayesian reinforcement learning algorithms, is studied when the model class is misspecified. A complete dynamic classification of posterior evolution in a misspecified two-armed Gaussian bandit is provided, identifying distinct regimes: correct model concentration, incorrect model concentration, and persistent belief mixing, characterized by the direction of statistical evidence and the model-action mapping. These regimes yield sharp predictions for limiting beliefs, action frequencies, and asymptotic regret. The analysis is then extended to a general finite model class and a unified stochastic stability framework is developed that represents posterior evolution as a Markov process on the belief simplex. This approach characterizes two sufficient conditions to classify the ergodic and transient behaviors and provides inductive dimensional reductions of the posterior dynamics. The results offer the first qualitative and geometric classification of TS under misspecification, bridging Bayesian learning with evolutionary dynamics, and build the foundations of robust decision-making in structured bandits.
