A0632
Title: Learning with importance weighted variational inference
Authors: Kamelia Daudel - ESSEC Business School (France) [presenting]
Abstract: Several popular variational bounds involving importance weighting ideas have been proposed to generalize and improve on the Evidence Lower BOund (ELBO) in the context of maximum likelihood optimization, such as the importance weighted auto-encoder (IWAE) and the variational Renyi (VR) bounds. The methodology to learn the parameters of interest with these bounds typically amounts to running stochastic gradient-based variational inference algorithms that incorporate the reparameterization trick. While the outcome of the resulting variational inference algorithms is expected to be tied to the choice of the variational bound, the precise effect of that choice remains poorly understood. The comparison of the ELBO, IWAE, and VR bounds methodologies is enabled by providing asymptotic analyses for key stochastic gradient estimators used in these methodologies. The analyses also reveal how these estimators compare to each other, and the theoretical findings are empirically illustrated.
