B0631
Title: Dynamics of mean-field approximation: A case-study in singular models
Authors: Anirban Bhattacharya - Texas AM University (United States) [presenting]
Debdeep Pati - Florida State University (United States)
Yun Yang - University of Illinois Urbana-Champaign (United States)
Sean Plummer - University of Illinois Urbana-Champaign (United States)
Abstract: The marginal likelihood or evidence in Bayesian statistics contains an intrinsic penalty for larger model sizes and is a fundamental quantity in Bayesian model comparison. Over the past two decades, there has been steadily increasing activity to understand the nature of this penalty in singular statistical models, building on pioneering work by Sumio Watanabe. Unlike regular models where the Bayesian information criterion (BIC) encapsulates a first-order expansion of the logarithm of the marginal likelihood, parameter counting gets trickier in singular models where a quantity called the real log canonical threshold (RLCT) summarizes the effective model dimensionality. We show that mean-field variational inference correctly recovers the RLCT for any singular model in its canonical or normal form. We additionally exhibit the sharpness of our bound by analyzing the dynamics of a general-purpose coordinate ascent algorithm (CAVI) popularly employed in variational inference.
