OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

• DOI: 10.48550/arxiv.2306.11249
• Publication Date: 2023-01-01
• Authors: Cheng Tan; Siyuan Li; Zhangyang Gao; Wenfei Guan; Zedong Wang; Zicheng Liu; Lirong Wu; Stan Z. Li
• Keywords: Benchmark (surveying); Predictive modelling; Code (set theory)

Spatio-temporal predictive learning is a paradigm that enables models to learn spatial and temporal patterns by predicting future frames from given past in an unsupervised manner. Despite remarkable progress recent years, lack of systematic understanding persists due the diverse settings, complex implementation, difficult reproducibility. Without standardization, comparisons can be unfair insights inconclusive. To address this dilemma, we propose OpenSTL, comprehensive benchmark for spatio-temporal categorizes prevalent approaches into recurrent-based recurrent-free models. OpenSTL provides modular extensible framework implementing various state-of-the-art methods. We conduct standard evaluations on datasets across domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow weather forecasting. Based our observations, provide detailed analysis how model architecture dataset properties affect performance. Surprisingly, find achieve good balance between efficiency performance than recurrent Thus, further extend common MetaFormers boost spatial-temporal learning. open-source code at https://github.com/chengtan9907/OpenSTL.