Learning stochastic dynamics with statistics-informed neural network

• DOI: 10.1016/j.jcp.2022.111819
• Publication Date: 2022-12-05T16:56:59Z
• Authors: Yuanran Zhu; Yu-Hang Tang; Changho Kim
• Keywords: FOS: Computer and information sciences; Computer Science - Machine Learning; Computer Science - Artificial Intelligence; Recurrent neural network; Mathematical sciences; FOS: Physical sciences; Dynamical Systems (math.DS); 01 natural sciences; Mathematical Sciences; Machine Learning (cs.LG); Engineering; 0103 physical sciences; FOS: Mathematics; Mathematics - Dynamical Systems; Condensed Matter - Statistical Mechanics; Reduced-order stochastic modeling; Scientific machine learning; Statistical Mechanics (cond-mat.stat-mech); Rare events; Applied Mathematics; Computational Physics (physics.comp-ph); Physical sciences; Artificial Intelligence (cs.AI); Physical Sciences; Physics - Computational Physics

We introduce a machine-learning framework named statistics-informed neural network (SINN) for learning stochastic dynamics from data. This new architecture was theoretically inspired by a universal approximation theorem for stochastic systems, which we introduce in this paper, and the projection-operator formalism for stochastic modeling. We devise mechanisms for training the neural network model to reproduce the correct \emph{statistical} behavior of a target stochastic process. Numerical simulation results demonstrate that a well-trained SINN can reliably approximate both Markovian and non-Markovian stochastic dynamics. We demonstrate the applicability of SINN to coarse-graining problems and the modeling of transition dynamics. Furthermore, we show that the obtained reduced-order model can be trained on temporally coarse-grained data and hence is well suited for rare-event simulations.