Transformers have achieved superior performances in many tasks natural language processing and computer vision, which also triggered great interest the time series community. Among multiple advantages of Transformers, ability to capture long-range dependencies interactions is especially attractive for modeling, leading exciting progress various applications. In this paper, we systematically review Transformer schemes modeling by highlighting their strengths as well limitations. particular,...

Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, unable to capture the global view of time (e.g. overall trend). To address these problems, we propose combine Transformer with seasonal-trend decomposition method, in which method captures profile while Transformers detailed structures. further enhance performance prediction, exploit fact that most tend a...

Deep learning performs remarkably well on many time series analysis tasks recently. The superior performance of deep neural networks relies heavily a large number training data to avoid overfitting. However, the labeled real-world applications may be limited such as classification in medical and anomaly detection AIOps. As an effective way enhance size quality data, augmentation is crucial successful application models data. In this paper, we systematically review different methods for...

Decomposing complex time series into trend, seasonality, and remainder components is an important task to facilitate anomaly detection forecasting. Although numerous methods have been proposed, there are still many characteristics exhibiting in real-world data which not addressed properly, including 1) ability handle seasonality fluctuation shift, abrupt change trend reminder; 2) robustness on with anomalies; 3) applicability long period. In the paper, we propose a novel generic...

Time series are the primary data type used to record dynamic system measurements and generated in great volume by both physical sensors online processes (virtual sensors). analytics is therefore crucial unlocking wealth of information implicit available data. With recent advancements graph neural networks (GNNs), there has been a surge GNN-based approaches for time analysis. These can explicitly model inter-temporal inter-variable relationships, which traditional other deep network-based...

Air pollution is a crucial issue affecting human health and livelihoods, as well one of the barriers to economic growth. Forecasting air quality has become an increasingly important endeavor with significant social impacts, especially in emerging countries. In this paper, we present novel Transformer termed AirFormer predict nationwide China, unprecedented fine spatial granularity covering thousands locations. decouples learning process into two stages: 1) bottom-up deterministic stage that...

Time series anomaly detection is critical for a wide range of applications. It aims to identify deviant samples from the normal sample distribution in time series. The most fundamental challenge this task learn representation map that enables effective discrimination anomalies. Reconstruction-based methods still dominate, but learning with anomalies might hurt performance its large abnormal loss. On other hand, contrastive find can clearly distinguish any instance others, which bring more...

Time series forecasting holds significant importance in many real-world dynamic systems and has been extensively studied. Unlike natural language process (NLP) computer vision (CV), where a single large model can tackle multiple tasks, models for time are often specialized, necessitating distinct designs different tasks applications. While pre-trained foundation have made impressive strides NLP CV, their development domains constrained by data sparsity. Recent studies revealed that (LLMs)...

Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence labeled data. Based pre-training and fine-tuning strategy, even a small amount data can achieve high performance. Compared with many published self-supervised surveys computer vision natural language processing, comprehensive survey for still missing. To fill this gap, we review current state-of-the-art methods in...

Time series analysis stands as a focal point within the data mining community, serving cornerstone for extracting valuable insights crucial to myriad of real-world applications. Recent advances in Foundation Models (FMs) have fundamentally reshaped paradigm model design time analysis, boosting various downstream tasks practice. These innovative approaches often leverage pre-trained or fine-tuned FMs harness generalized knowledge tailored analysis. This survey aims furnish comprehensive and...

The monitoring and management of numerous diverse time series data at Alibaba Group calls for an effective scalable anomaly detection service. In this paper, we propose RobustTAD, a Robust Time Anomaly Detection framework by integrating robust seasonal-trend decomposition convolutional neural network data. can effectively handle complicated patterns in series, meanwhile significantly simplifies the architecture network, which is encoder-decoder with skip connections. This capture multi-scale...

Recent studies have shown that deep learning models such as RNNs and Transformers brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information. We found, however, there is still great room improvement in how to preserve information neural networks while avoiding overfitting noise presented the history. Addressing this allows better utilization capabilities models. To end, we design a \textbf{F}requency...

Periodicity detection is a crucial step in time series tasks, including monitoring and forecasting of metrics many areas, such as IoT applications self-driving database management system. In these applications, multiple periodic components exist are often interlaced with each other. Such dynamic complicated patterns make the accurate periodicity difficult. addition, other series, trend, outliers noises, also pose additional challenges for detection. this paper, we propose robust general...

Time series anomaly detection is a challenging problem due to the complex temporal dependencies and limited label data. Although some algorithms including both traditional deep models have been proposed, most of them mainly focus on time-domain modeling, do not fully utilize information in frequency domain time In this paper, we propose Time-Frequency analysis based Anomaly Detection model, or TFAD for short, exploit domains performance improvement. Besides, incorporate decomposition data...

Time series forecasting is a critical and challenging problem in many real applications. Recently, Transformer-based models prevail time due to their advancement long-range dependencies learning. Besides, some introduce decomposition further unveil reliable yet plain temporal dependencies. Unfortunately, few could handle complicated periodical patterns, such as multiple periods, variable phase shifts real-world datasets. Meanwhile, the notorious quadratic complexity of dot-product attentions...

Electric load forecasting is an essential problem for the power industry, which has a significant impact on system operation. Currently, deep learning proved to be effective tool forecasting. However, those learning-based models are vulnerable towards adversarial attacks, raises concerns about robustness of models. In this study, we propose Bayesian training method enhance attacks. We theoretically prove that proposed can improve against various attacking objectives without compromising...

Spatio-temporal graph neural networks (STGNN) have emerged as the dominant model for spatio-temporal (STG) forecasting. Despite their success, they fail to intrinsic uncertainties within STG data, which cripples practicality in downstream tasks decision-making. To this end, paper focuses on probabilistic forecasting, is challenging due difficulty modeling and complex ST dependencies. In study, we present first attempt generalize popular de-noising diffusion models STGs, leading a novel...

Temporal data, notably time series and spatio-temporal are prevalent in real-world applications. They capture dynamic system measurements produced vast quantities by both physical virtual sensors. Analyzing these data types is vital to harnessing the rich information they encompass thus benefits a wide range of downstream tasks. Recent advances large language other foundational models have spurred increased use mining. Such methodologies not only enable enhanced pattern recognition reasoning...

Transformer-based models have achieved some success in time series forecasting. Existing methods mainly model from limited or fixed scales, making it challenging to capture different characteristics spanning various scales. In this paper, we propose multi-scale transformers with adaptive pathways (Pathformer). The proposed Transformer integrates both temporal resolution and distance for modeling. Multi-scale division divides the into resolutions using patches of sizes. Based on each scale,...

The ubiquitous missing values cause the multivariate time series data to be partially observed, destroying integrity of and hindering effective analysis. Recently deep learning imputation methods have demonstrated remarkable success in elevating quality corrupted data, subsequently enhancing performance downstream tasks. In this paper, we conduct a comprehensive survey on recently proposed methods. First, propose taxonomy for reviewed methods, then provide structured review these by...

Accurate weather forecasts are important for disaster prevention, agricultural planning, and water resource management. Traditional numerical prediction (NWP) methods offer physically interpretable high-accuracy predictions but computationally expensive fail to fully leverage rapidly growing historical data. In recent years, deep learning have made significant progress in forecasting, challenges remain, such as balancing global regional high-resolution forecasts, excessive smoothing extreme...

Many real-world time series data exhibit complex patterns with trend, seasonality, outlier and noise. Robustly accurately decomposing these components would greatly facilitate tasks including anomaly detection, forecasting classification. RobustSTL is an effective seasonal-trend decomposition for complicated patterns. However, it cannot handle multiple seasonal properly. Also suffers from its high computational complexity, which limits usage in practice. In this paper, we extend to...

Anomalies are ubiquitous in real-world time-series data which call for effective and timely detection, especially an unsupervised setting labeling cost saving. In this paper, we develop density reconstruction model multi-dimensional anomaly detection. particular, it directly handles important realistic that the detection is achieved towards raw contaminated with noise training, contrast to most existing works assume training general clean i.e. not anomaly. It extends recent advancements deep...

Transformers have achieved superior performances in many tasks natural language processing and computer vision, which also triggered great interest the time series community. Among multiple advantages of Transformers, ability to capture long-range dependencies interactions is especially attractive for modeling, leading exciting progress various applications. In this paper, we systematically review Transformer schemes modeling by highlighting their strengths as well limitations. particular,...

The advent of Large Language Models (LLMs) has brought in a new era possibilities the realm education. This survey paper summarizes various technologies LLMs educational settings from multifaceted perspectives, encompassing student and teacher assistance, adaptive learning, commercial tools. We systematically review technological advancements each perspective, organize related datasets benchmarks, identify risks challenges associated with deploying Furthermore, we outline future research...