Mathematical analysis of singularities in the diffusion model under the submanifold assumption

• DOI: 10.48550/arxiv.2301.07882
• Publication Date: 2023-01-01
• Authors: Yibin Lu; Zhongjian Wang; Guillaume Bal
• Keywords: Conditional expectation

This paper provide several mathematical analyses of the diffusion model in machine learning. The drift term backwards sampling process is represented as a conditional expectation involving data distribution and forward diffusion. training aims to find such function by minimizing mean-squared residue related expectation. Using small-time approximations Green's diffusion, we show that analytical mean DDPM score SGM asymptotically blow up final stages for singular distributions those concentrated on lower-dimensional manifolds, therefore difficult approximate network. To overcome this difficulty, derive new target associated loss, which remains bounded even distributions. We illustrate theoretical findings with numerical examples.