Hybrid Classifiers for Spatio-temporal Real-time Abnormal Behaviors Detection, Tracking, and Recognition in Massive Hajj Crowds

• DOI: 10.48550/arxiv.2207.11931
• Publication Date: 2022-01-01
• Authors: Tarik Alafif; Anas A. Hadi; Manal Allahyani; Bander Alzahrani; Areej Alhothali; Reem Alotaibi; Ahmed Barnawi
• Keywords: Crowd psychology; Abnormality

Individual abnormal behaviors vary depending on crowd sizes, contexts, and scenes. Challenges such as partial occlusions, blurring, large-number behavior, camera viewing occur in large-scale crowds when detecting, tracking, recognizing individuals with behaviors. In this paper, our contribution is twofold. First, we introduce an annotated labeled Hajj dataset (HAJJv2). Second, propose two methods of hybrid Convolutional Neural Networks (CNNs) Random Forests (RFs) to detect recognize Spatio-temporal small large-scales videos. small-scale videos, a ResNet-50 pre-trained CNN model fine-tuned verify whether every frame normal or the spatial domain. If anomalous are observed, motion-based detection method based magnitudes orientations Horn-Schunck optical flow used locate track A Kalman filter employed videos predict detected subsequent frames. Then, means, variances, standard deviations statistical features computed fed RF classify temporal crowds, fine-tune using YOLOv2 object technique