A0827
Title: Adaptivity of diffusion models to manifold structures
Authors: Rong Tang - HKUST (Hong Kong)
Yun Yang - University of Illinois Urbana-Champaign (United States) [presenting]
Abstract: Empirical studies have demonstrated the effectiveness of (score-based) diffusion models in generating high-dimensional data, such as texts and images, which typically exhibit a low-dimensional manifold nature. These empirical successes raise the theoretical question of whether score-based diffusion models can optimally adapt to low-dimensional manifold structures. While recent work has validated the minimax optimality of diffusion models when the target distribution admits a smooth density with respect to the Lebesgue measure of the ambient data space, these findings do not fully account for the ability of diffusion models to avoid the curse of dimensionality when estimating high-dimensional distributions. The aim is to consider two common classes of diffusion models: Langevin diffusion and forward-backwards diffusion. Both models can adapt to the intrinsic manifold structure by showing that the convergence rate of the inducing distribution estimator depends only on the intrinsic dimension of the data. Moreover, the considered estimator does not require knowing or explicitly estimating the manifold. It is also demonstrated that the forward-backwards diffusion can achieve the minimax optimal rate under the Wasserstein metric when the target distribution possesses a smooth density with respect to the volume measure of the low-dimensional manifold.
