Deep Neural Network for Visual Stimulus-Based Reaction Time Estimation Using the Periodogram of Single-Trial EEG

• DOI: 10.3390/s20216090
• Publication Date: 2020-10-27T13:22:45Z
• Authors: Mohammad Samin Nu...; Arindam Dutta; Matthew Kyle Robison; Chris Blais; Gene Arnold Brewer; Daniel Wesley Bliss
• Keywords: reaction time; EEG signal processing; Chemical technology; brain–computer interface; deep learning; Electroencephalography; Fully Connected Neural Network (FCNN); TP1-1185; periodogram; Convolutional Neural Network (CNN); Article; 03 medical and health sciences; 0302 clinical medicine; Reaction Time; Humans; Neural Networks, Computer; Algorithms

Multiplexed deep neural networks (DNN) have engendered high-performance predictive models gaining popularity for decoding brain waves, extensively collected in the form of electroencephalogram (EEG) signals. In this paper, to the best of our knowledge, we introduce a first-ever DNN-based generalized approach to estimate reaction time (RT) using the periodogram representation of single-trial EEG in a visual stimulus-response experiment with 48 participants. We have designed a Fully Connected Neural Network (FCNN) and a Convolutional Neural Network (CNN) to predict and classify RTs for each trial. Though deep neural networks are widely known for classification applications, cascading FCNN/CNN with the Random Forest model, we designed a robust regression-based estimator to predict RT. With the FCNN model, the accuracies obtained for binary and 3-class classification were 93% and 76%, respectively, which further improved with the use of CNN (94% and 78%, respectively). The regression-based approach predicted RTs with correlation coefficients (CC) of 0.78 and 0.80 for FCNN and CNN, respectively. Investigating further, we found that the left central as well as parietal and occipital lobes were crucial for predicting RT, with significant activities in the theta and alpha frequency bands.