A0435
Title: Semi-implicit variational inference with score matching
Authors: Cheng Zhang - Peking University (China) [presenting]
Abstract: Semi-implicit variational inference (SIVI) greatly enriches the expressiveness of variational families by considering implicit variational distributions defined in a hierarchical manner. However, due to the intractable densities of semi-implicit distributions, typical SIVI approaches often use surrogate evidence lower bounds (ELBOs) or employ expensive inner-loop MCMC runs for unbiased ELBOs for training. New SIVI methods are introduced based on several alternative training objectives via score matching, which allows the leverage of the hierarchical structure of semi-implicit distributions to bypass the intractability of their densities. The basic score-matching framework for SIVI is started with a minimax formulation called SIVI-SM. How to further enhance the flexibility of semi-implicit distribution is then discussed by allowing multiple hierarchical layers, which can also be used to accelerate the diffusion model given the learned score networks. Lastly, KSIVI is introduced, a variant of SIVI-SM that eliminates the need for lower-level optimization through kernel tricks. An upper bound for the variance of the Monte Carlo gradient estimators of the KSD objective is derived, which allows establishing novel convergence guarantees of KSIVI.
