(Publisher of Peer Reviewed Open Access Journals)

International Journal of Advanced Technology and Engineering Exploration (IJATEE)

ISSN (Print):2394-5443    ISSN (Online):2394-7454
Volume-10 Issue-100 March-2023
Full-Text PDF
Paper Title : A deep learning approach to detect the electroencephalogram-based cognitive task states
Author Name : Hitesh Yadav and Surita Maini
Abstract :

Cognitive abilities are responsible for performing various simple and complex activities that affect a person's mental performance. These are also responsible for different day-to-day actions in human life. In the past few years, studies on cognitive ability, mental performance, and mindfulness meditation have been seen more frequently. The electroencephalogram (EEG) is an effective technique to study brain dynamics while executing any cognitive task and leads to new possibilities in the brain-computer interface (BCI) field. In this study, twenty-seven (27) healthy subjects performed a designed cognitive task having three different states (i.e., rest, meditation, and arithmetic) to stimulate the brain's cognitive functions. BIOPAC-MP-160 has been used for the EEG signal acquisition of the designed cognitive task according to the international 10-20 data acquisition system. The EEGLAB has been used to visualize, pre-process, filter, and removal of noise from the data. Then phase-amplitude coupling is performed to extract the features. After completing the feature extraction, the classification has been performed by three different deep learning approaches, i.e., sequential convolutional network (SCN), multi-branch convolutional network (MBCN), and multi-branch convolutional network-bidirectional long short-term memory network (MBCN-Bi-LSTM). The performance of the different classifications model has been estimated in terms of accuracy, precision, F1 score, and recall. The results demonstrated that MBCN-Bi-LSTM performs better than the SCN and MBCN, with a significant improvement in accuracy of 97.99%. The comparative analysis of the previously used deep learning and machine learning approaches to classify the EEG signal of different brain states substantially indicates that the proposed MBCN-Bi-LSTM model performs better in terms of accuracy and error rate. Also, the computational execution time of the proposed MBCN-Bi-LSTM is found to be less than the previous methods. The proposed classification approach may be utilized in future research to classify the various physiological signals.

Keywords : BCI, EEG, Deep learning, Classification, Cognitive task, Mental state classification.
Cite this article : Yadav H, Maini S. A deep learning approach to detect the electroencephalogram-based cognitive task states. International Journal of Advanced Technology and Engineering Exploration. 2023; 10(100):303-320. DOI:10.19101/IJATEE.2021.876520.
References :
[1]Vansteensel MJ, Pels EG, Bleichner MG, Branco MP, Denison T, Freudenburg ZV et al. Fully implanted brain–computer interface in a locked-in patient with ALS. New England Journal of Medicine. 2016; 375(21):2060-6.
[Crossref] [Google Scholar]
[2]Wang H, Chang W, Zhang C. Functional brain network and multichannel analysis for the P300-based brain computer interface system of lying detection. Expert Systems with Applications. 2016; 53:117-28.
[Crossref] [Google Scholar]
[3]Fujiwara K, Abe E, Kamata K, Nakayama C, Suzuki Y, Yamakawa T, et al. Heart rate variability-based driver drowsiness detection and its validation with EEG. IEEE Transactions on Biomedical Engineering. 2018; 66(6):1769-78.
[Crossref] [Google Scholar]
[4]Gao X, Wang Y, Chen X, Gao S. Interface, interaction, and intelligence in generalized brain–computer interfaces. Trends in Cognitive Sciences. 2021; 25(8):671-84.
[Crossref] [Google Scholar]
[5]Hramov AE, Maksimenko VA, Pisarchik AN. Physical principles of brain–computer interfaces and their applications for rehabilitation, robotics and control of human brain states. Physics Reports. 2021; 918:1-33.
[Crossref] [Google Scholar]
[6]Yadav D, Yadav S, Veer K. A comprehensive assessment of brain computer interfaces: recent trends and challenges. Journal of Neuroscience Methods. 2020; 346(2020):1-20.
[Crossref] [Google Scholar]
[7]Gu X, Cao Z, Jolfaei A, Xu P, Wu D, Jung TP, et al. EEG-based brain-computer interfaces (BCIs): A survey of recent studies on signal sensing technologies and computational intelligence approaches and their applications. IEEE/ACM Transactions on Computational Biology and Bioinformatics. 2021; 18(5):1645-66.
[Crossref] [Google Scholar]
[8]Bagheri M, Power SD. Simultaneous classification of both mental workload and stress level suitable for an online passive brain–computer interface. Sensors. 2022; 22(2):1-17.
[Crossref] [Google Scholar]
[9]Lohani M, Payne BR, Strayer DL. A review of psychophysiological measures to assess cognitive states in real-world driving. Frontiers in Human Neuroscience. 2019; 13:1-27.
[Crossref] [Google Scholar]
[10]Kongwudhikunakorn S, Kiatthaveephong S, Thanontip K, Leelaarporn P, Piriyajitakonkij M, Charoenpattarawut T, et al. A pilot study on visually stimulated cognitive tasks for EEG-based dementia recognition. IEEE Transactions on Instrumentation and Measurement. 2021; 70:1-10.
[Crossref] [Google Scholar]
[11]Sharma S, Singh G, Sharma M. A comprehensive review and analysis of supervised-learning and soft computing techniques for stress diagnosis in humans. Computers in Biology and Medicine. 2021; 134(2021):1-19.
[Crossref] [Google Scholar]
[12]Keil A, Bernat EM, Cohen MX, Ding M, Fabiani M, Gratton G, et al. Recommendations and publication guidelines for studies using frequency domain and time‐frequency domain analyses of neural time series. Psychophysiology. 2022; 59(5):1-37.
[Crossref] [Google Scholar]
[13]Nafea MS, Ismail ZH. Supervised machine learning and deep learning techniques for epileptic seizure recognition using EEG signals-a systematic literature review. Bioengineering. 2022; 9(12):1-35.
[Crossref] [Google Scholar]
[14]Wang J, Wang M. Review of the emotional feature extraction and classification using EEG signals. Cognitive Robotics. 2021; 1:29-40.
[Crossref] [Google Scholar]
[15]Zhou Y, Xu Z, Niu Y, Wang P, Wen X, Wu X, et al. Cross-task cognitive workload recognition based on EEG and domain adaptation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2022; 30:50-60.
[Crossref] [Google Scholar]
[16]Zheng X, Chen W, You Y, Jiang Y, Li M, Zhang T. Ensemble deep learning for automated visual classification using EEG signals. Pattern Recognition. 2020; 102:1-33.
[Crossref] [Google Scholar]
[17]Li G, Lee CH, Jung JJ, Youn YC, Camacho D. Deep learning for EEG data analytics: a survey. Concurrency and Computation: Practice and Experience. 2020; 32(18):1-13.
[Crossref] [Google Scholar]
[18]Zhao X, Zhang H, Zhu G, You F, Kuang S, Sun L. A multi-branch 3D convolutional neural network for EEG-based motor imagery classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2019; 27(10):2164-77.
[Crossref] [Google Scholar]
[19]Qiao W, Bi X. Ternary-task convolutional bidirectional neural turing machine for assessment of EEG-based cognitive workload. Biomedical Signal Processing and Control. 2020; 57:1-13.
[Crossref] [Google Scholar]
[20]Abbasi SF, Ahmad J, Tahir A, Awais M, Chen C, Irfan M, et al. EEG-based neonatal sleep-wake classification using multilayer perceptron neural network. IEEE Access. 2020; 8:183025-34.
[Crossref] [Google Scholar]
[21]Ha KW, Jeong JW. Motor imagery EEG classification using capsule networks. Sensors. 2019; 19(13):1-20.
[Crossref] [Google Scholar]
[22]Saccá V, Campolo M, Mirarchi D, Gambardella A, Veltri P, Morabito FC. On the classification of EEG signal by using an SVM based algorithm. Multidisciplinary Approaches to Neural Computing. 2018:271-8.
[Crossref] [Google Scholar]
[23]Suchetha M, Madhumitha R, Sruthi R. Sequential convolutional neural networks for classification of cognitive tasks from EEG signals. Applied Soft Computing. 2021; 111(2021):1-14.
[Crossref] [Google Scholar]
[24]Al-fahoum AS, Al-fraihat AA. Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains. International Scholarly Research Notices. 2014; 2014:1-8.
[Crossref] [Google Scholar]
[25]Scally B, Burke MR, Bunce D, Delvenne JF. Resting-state EEG power and connectivity are associated with alpha peak frequency slowing in healthy aging. Neurobiology of Aging. 2018; 71:149-55.
[Crossref] [Google Scholar]
[26]Jacobs D, Hilton T, Del CM, Carlen PL, Bardakjian BL. Classification of pre-clinical seizure states using scalp EEG cross-frequency coupling features. IEEE Transactions on Biomedical Engineering. 2018; 65(11):2440-9.
[Crossref] [Google Scholar]
[27]Nuamah JK, Seong Y. Support vector machine (SVM) classification of cognitive tasks based on electroencephalography (EEG) engagement index. Brain-Computer Interfaces. 2018; 5(1):1-12.
[Crossref] [Google Scholar]
[28]Dutta S, Singh M, Kumar A. Automated classification of non-motor mental task in electroencephalogram based brain-computer interface using multivariate autoregressive model in the intrinsic mode function domain. Biomedical Signal Processing and Control. 2018; 43:174-82.
[Crossref] [Google Scholar]
[29]Noshadi S, Abootalebi V, Sadeghi MT, Shahvazian MS. Selection of an efficient feature space for EEG-based mental task discrimination. Biocybernetics and Biomedical Engineering. 2014; 34(3):159-68.
[Crossref] [Google Scholar]
[30]Gupta A, Agrawal RK, Kirar JS, Andreu-perez J, Ding WP, Lin CT, et al. On the utility of power spectral techniques with feature selection techniques for effective mental task classification in noninvasive BCI. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2019; 51(5):3080-92.
[Crossref] [Google Scholar]
[31]Gaurav AR, Kumar V. EEG-metric based mental stress detection. Network Biology. 2018; 8(1):25-34.
[Google Scholar]
[32]Qayyum A, Khan MA, Mazher M, Suresh M. Classification of EEG learning and resting states using 1d-convolutional neural network for cognitive load assessment. In student conference on research and development (SCOReD) 2018 (pp. 1-5). IEEE.
[Crossref] [Google Scholar]
[33]Zhang P, Wang X, Zhang W, Chen J. Learning spatial–spectral–temporal EEG features with recurrent 3D convolutional neural networks for cross-task mental workload assessment. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2018; 27(1):31-42.
[Crossref] [Google Scholar]
[34]Jiménez-guarneros M, Gómez-gil P. Custom domain adaptation: a new method for cross-subject, EEG-based cognitive load recognition. IEEE Signal Processing Letters. 2020; 27:750-4.
[Crossref] [Google Scholar]
[35]Idowu OP, Ilesanmi AE, Li X, Samuel OW, Fang P, Li G. An integrated deep learning model for motor intention recognition of multi-class EEG signals in upper limb amputees. Computer Methods and Programs in Biomedicine. 2021; 206:1-39.
[Crossref] [Google Scholar]
[36]Liu Q, Cai J, Fan SZ, Abbod MF, Shieh JS, Kung Y, et al. Spectrum analysis of EEG signals using CNN to model patient’s consciousness level based on anesthesiologists’ experience. IEEE Access. 2019; 7:53731-42.
[Crossref] [Google Scholar]
[37]Schirrmeister RT, Springenberg JT, Fiederer LD, Glasstetter M, Eggensperger K, Tangermann M, et al. Deep learning with convolutional neural networks for EEG decoding and visualization. Human Brain Mapping. 2017; 38(11):5391-420.
[Crossref] [Google Scholar]
[38]Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. Journal of Neural Engineering. 2018; 15(5):1-30.
[Crossref] [Google Scholar]
[39]Yang Y, Wu Q, Qiu M, Wang Y, Chen X. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network. In international joint conference on neural networks 2018 (pp. 1-7). IEEE.
[Crossref] [Google Scholar]
[40]Bird JJ, Kobylarz J, Faria DR, Ekárt A, Ribeiro EP. Cross-domain MLP and CNN transfer learning for biological signal processing: EEG and EMG. IEEE Access. 2020; 8:54789-801.
[Crossref] [Google Scholar]
[41]Samanta K, Chatterjee S, Bose R. Cross-subject motor imagery tasks EEG signal classification employing multiplex weighted visibility graph and deep feature extraction. IEEE Sensors Letters. 2019; 4(1):1-4.
[Crossref] [Google Scholar]
[42]Kwon S. CLSTM: deep feature-based speech emotion recognition using the hierarchical ConvLSTM network. Mathematics. 2020; 8(12):1-19.
[Crossref] [Google Scholar]
[43]Sajjad M, Kwon S. Clustering-based speech emotion recognition by incorporating learned features and deep BiLSTM. IEEE Access. 2020; 8:79861-75.
[Crossref] [Google Scholar]
[44]Roy Y, Banville H, Albuquerque I, Gramfort A, Falk TH, Faubert J. Deep learning-based electroencephalography analysis: a systematic review. Journal of Neural Engineering. 2019; 16(5):1-38.
[Crossref] [Google Scholar]
[45]Amin SU, Alsulaiman M, Muhammad G, Mekhtiche MA, Hossain MS. Deep learning for EEG motor imagery classification based on multi-layer CNNs feature fusion. Future Generation Computer Systems. 2019; 101:542-54.
[Crossref] [Google Scholar]
[46]Altuwaijri GA, Muhammad G. A multibranch of convolutional neural network models for electroencephalogram-based motor imagery classification. Biosensors. 2022; 12(1):1-21.
[Crossref] [Google Scholar]
[47]Farsi L, Siuly S, Kabir E, Wang H. Classification of alcoholic EEG signals using a deep learning method. IEEE Sensors Journal. 2020; 21(3):3552-60.
[Crossref] [Google Scholar]
[48]Ambrosini E, Vallesi A. Asymmetry in prefrontal resting-state EEG spectral power underlies individual differences in phasic and sustained cognitive control. Neuroimage. 2016; 124:843-57.
[Crossref] [Google Scholar]
[49]Gong R, Wegscheider M, Mühlberg C, Gast R, Fricke C, Rumpf JJ, et al. Spatiotemporal features of β-γ phase-amplitude coupling in Parkinson’s disease derived from scalp EEG. Brain. 2021; 144(2):487-503.
[Crossref] [Google Scholar]
[50]Jurkiewicz GJ, Hunt MJ, Żygierewicz J. Addressing pitfalls in phase-amplitude coupling analysis with an extended modulation index toolbox. Neuroinformatics. 2021; 19:319-45.
[Crossref] [Google Scholar]
[51]Tort AB, Komorowski R, Eichenbaum H, Kopell N. Measuring phase-amplitude coupling between neuronal oscillations of different frequencies. Journal of Neurophysiology. 2010; 104(2):1195-210.
[Crossref] [Google Scholar]
[52]Hülsemann MJ, Naumann E, Rasch B. Quantification of phase-amplitude coupling in neuronal oscillations: comparison of phase-locking value, mean vector length, modulation index, and generalized-linear-modeling-cross-frequency-coupling. Frontiers in Neuroscience. 2019; 13:1-15.
[Crossref] [Google Scholar]
[53]Crandall B, Klein GA, Hoffman RR. Working minds: a practitioners guide to cognitive task analysis. Mit Press; 2006.
[Google Scholar]
[54]Salmon P, Stanton NA, Gibbon A, Jenkins D, Walker GH. Human factors methods and sports science: a practical guide. CRC Press; 2009.
[Google Scholar]
[55]Selim S, Tantawi MM, Shedeed HA, Badr A. A csp am-ba-svm approach for motor imagery BCI system. IEEE Access. 2018; 6:49192-208.
[Crossref] [Google Scholar]
[56]Sreeja SR, Samanta D. Classification of multiclass motor imagery EEG signal using sparsity approach. Neurocomputing. 2019; 368:133-45.
[Crossref] [Google Scholar]
[57]Sun Y, Xue B, Zhang M, Yen GG. Completely automated CNN architecture design based on blocks. IEEE transactions on neural networks and learning systems. 2019; 31(4):1242-54.
[Crossref] [Google Scholar]
[58]Ma J, Cheng JC, Lin C, Tan Y, Zhang J. Improving air quality prediction accuracy at larger temporal resolutions using deep learning and transfer learning techniques. Atmospheric Environment. 2019; 214:1-10.
[Crossref] [Google Scholar]
[59]Hu L, Zhang Z. EEG signal processing and feature extraction. Singapore: Springer Singapore; 2019.
[Crossref] [Google Scholar]
[60]Fouladi S, Safaei AA, Mammone N, Ghaderi F, Ebadi MJ. Efficient deep neural networks for classification of Alzheimer’s disease and mild cognitive impairment from scalp EEG recordings. Cognitive Computation. 2022; 14(4):1247-68.
[Crossref] [Google Scholar]
[61]Siddiqui F, Mohammad A, Alam MA, Naaz S, Agarwal P, Sohail SS, et al. Deep neural network for EEG signal-based subject-independent imaginary mental task classification. Diagnostics. 2023; 13(4):1-23.
[Crossref] [Google Scholar]
[62]Wen D, Li R, Tang H, Liu Y, Wan X, Dong X, et al. Task-state EEG signal classification for spatial cognitive evaluation based on multiscale high-density convolutional neural network. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2022; 30:1041-51.
[Crossref] [Google Scholar]
[63]Cui Z, Zheng X, Shao X, Cui L. Automatic sleep stage classification based on convolutional neural network and fine-grained segments. Complexity. 2018; 2018:1-14.
[Crossref] [Google Scholar]
[64]Michielli N, Acharya UR, Molinari F. Cascaded LSTM recurrent neural network for automated sleep stage classification using single-channel EEG signals. Computers in Biology and Medicine. 2019; 106:71-81.
[Crossref] [Google Scholar]
[65]Nagabushanam P, Thomas GS, Radha S. EEG signal classification using LSTM and improved neural network algorithms. Soft Computing. 2020; 24:9981-10003.
[Crossref] [Google Scholar]
[66]Dvorak D, Shang A, Abdel-baki S, Suzuki W, Fenton AA. Cognitive behavior classification from scalp EEG signals. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2018; 26(4):729-39.
[Crossref] [Google Scholar]
[67]Yıldırım Ö, Baloglu UB, Acharya UR. A deep convolutional neural network model for automated identification of abnormal EEG signals. Neural Computing and Applications. 2020; 32:15857-68.
[Crossref] [Google Scholar]
[68]Ac Çİ, Kaya M, Mishchenko Y. Distinguishing mental attention states of humans via an EEG-based passive BCI using machine learning methods. Expert Systems with Applications. 2019; 134:153-66.
[Crossref] [Google Scholar]
[69]Toa CK, Sim KS, Tan SC. Electroencephalogram-based attention level classification using convolution attention memory neural network. IEEE Access. 2021; 9:58870-81.
[Crossref] [Google Scholar]
[70]Mohanavelu K, Poonguzhali S, Janani A, Vinutha S. Machine learning-based approach for identifying mental workload of pilots. Biomedical Signal Processing and Control. 2022.
[Crossref] [Google Scholar]