A0170
Title: A leave-one-out approach to approximate message passing
Authors: Zhigang Bao - Hong Kong University of Science and Technology (Hong Kong)
Qiyang Han - Rutgers University (United States)
Xiaocong Xu - Hong Kong University of Science and Technology (Hong Kong) [presenting]
Abstract: The aim is to present a non-asymptotic leave-one-out representation for approximate message passing (AMP) iterates, applicable to a broad class of Gaussian random matrix models with general variance profiles. In contrast to the typical AMP theory that describes the empirical distributions of the AMP iterate via a low-dimensional state evolution, the leave-one-out representation yields an intrinsically high-dimensional state evolution formula which provides non-asymptotic characterizations for the possibly heterogeneous, entrywise behaviour of the AMP iterate under the prescribed random matrix models. To exemplify some distinct features of our AMP theory in applications, the precise stochastic behaviour of the ridge estimator is analyzed for independent and non-identically distributed observations in the context of regularized linear estimation, whose covariates exhibit general variance profiles. It is found that its finite-sample distribution is characterized via a weighted ridge estimator in a heterogeneous Gaussian sequence model. Notably, in contrast to the i.i.d. sampling scenario, the effective noise and regularization are now full-dimensional vectors determined via a high-dimensional system of equations. The method of proof differs significantly from the master conditioning approach. It relies on an inductive method that sheds light on the intricate cancellation scheme for the trace function of certain random matrix recursions associated with the AMP.
