Real-time feature selection technique with concept drift detection using adaptive micro-clusters for data stream mining

• DOI: 10.1016/j.knosys.2018.08.007
• Publication Date: 2018-08-09T17:02:31Z
• Authors: Mahmood Shakir Ha...; Frederic Stahl; Atta Badii
• Keywords: Concept Drift; Feature (linguistics); Tracking (education)

Data streams are unbounded, sequential data instances that are generated with high Velocity. Classifying sequential data instances is a very challenging problem in machine learning with applications in network intrusion detection, financial markets and applications requiring real-time sensor-networks-based situation assessment. Data stream classification is concerned with the automatic labelling of unseen instances from the stream in real-time. For this the classifier needs to adapt to concept drifts and can only have a single pass through the data if the stream is fast moving. This research paper presents work on a real-time pre-processing technique, in particular feature tracking. The feature tracking technique is designed to improve Data Stream Mining (DSM) classification algorithms by enabling and optimising real-time feature selection. The technique is based on tracking adaptive statistical summaries of the data and class label distributions, known as Micro-Clusters. Currently the technique is able to detect concept drifts and identify which features have been influential in the drift.