A two‐stage data cleansing method for bridge global positioning system monitoring data based on bi‐direction long and short term memory anomaly identification and conditional generative adversarial networks data repair

• DOI: 10.1002/stc.2993
• Publication Date: 2022-05-12T05:29:00Z
• Authors: Kang Yang; Youliang Ding; Huachen Jiang; Hanwei Zhao; Gan Luo
• Keywords: Data cleansing; Identification; Data set; Anomaly (physics); Dynamic data

Data cleansing is an essential approach for improving data quality. Therefore, it the key to avoiding false alarm of monitoring system due anomaly itself. consists two parts: identification and repair. However, current research on has mainly focused lacks efficient repair methods. The lies in sensor correlation models based mapping relationships between sensors. To obtain a good inter-sensor relationship model, first necessary exclude anomalous from training set used modeling. two-stage framework collaborative multi-sensor proposed. First, analysis features GPS data, bidirectional long- short-term memory (Bi-LSTM) neural network model adopted anomalies classification localization. As result, segment be repaired determined. Then, basis all time range day before target segment, constructed by excluding segments with help above results. conditional generation adversarial (CGAN) proposed achieve Experimental validation shows that method followed can accurately identify anomalies. Finally, several factors affecting effect are discussed.