(Publisher of Peer Reviewed Open Access Journals)

International Journal of Advanced Technology and Engineering Exploration (IJATEE)

ISSN (Print):2394-5443    ISSN (Online):2394-7454
Volume-11 Issue-115 June-2024
Full-Text PDF
Paper Title : Emotion recognition from EEG signal data of the brain using bidirectional long short-term memory
Author Name : Markapudi Sowmya, Pothuri Surendra Varma and Katarapu Deepika
Abstract :

This study aimed to develop an emotion recognition model using brain signals. The brain computer interface (BCI) focuses on creating technology that enables direct brain-to-external device connections. There are two forms of BCI: invasive and non-invasive. In BCI, electroencephalography (EEG) is essential. EEG is a non-invasive method that involves applying electrodes to the scalp to capture electrical activity in the brain. EEG data is utilized to decode the user's intended emotions, activities, and thoughts. Emotions are important for human interaction, communication, and overall well-being. Many paralyzed people worldwide are unable to express their emotions or meet their needs, making it difficult to understand them, which leads to feelings of isolation. However, it is possible to detect emotions using BCI. Emotions are reflected in electrical brain activity and can be analyzed using EEG signals. The EEG signals are then decoded to detect a person's respective emotions. The decoding process mainly includes three steps. First, the signals are pre-processed to remove noise, and data is encoded. Second, the relevant features are extracted using the spectral power method. Third, emotions are classified using long short-term memory (LSTM), gated recurrent unit (GRU), and bidirectional long short-term memory (BiLSTM) algorithms. New EEG data is given to the model, and then emotions are displayed. The model developed using BiLSTM achieved an accuracy of 93.97%. A comparison was made with existing classification techniques that have used many three-dimensional (3D) models and the arousal-valence ratio to identify a person's emotion. The model's generalization will improve further by testing it on different types of datasets. The model's generalization improves further by testing it on different types of datasets.

Keywords : EEG data, Brain computer interface, Pre-processing, Classification, Spectral power, BiLSTM, Emotions.
Cite this article : Sowmya M, Varma PS, Deepika K. Emotion recognition from EEG signal data of the brain using bidirectional long short-term memory. International Journal of Advanced Technology and Engineering Exploration. 2024; 11(115):930-942. DOI:10.19101/IJATEE.2023.10102441.
References :
[1]Huang X, Lajoie SP. Social emotional interaction in collaborative learning: why it matters and how can we measure it? Social Sciences & Humanities Open. 2023; 7(1):100447.
[Crossref] [Google Scholar]
[2]Sowmya B, Alex SA, Kanavalli A, Supreeth S, Shruthi G, Rohith S. Machine learning model for emotion detection and recognition using an enhanced convolutional neural network. Journal of Integrated Science and Technology. 2024; 12(4):1-10.
[Crossref] [Google Scholar]
[3]Kim H, Hong T. Enhancing emotion recognition using multimodal fusion of physiological, environmental, personal data. Expert Systems with Applications. 2024; 249:123723.
[Crossref] [Google Scholar]
[4]Kim H, Hong T. Enhancing emotion recognition using multimodal fusion of physiological, environmental, personal data. Expert Systems with Applications. 2024; 249:123723.
[Crossref] [Google Scholar]
[5]Shu L, Yu Y, Chen W, Hua H, Li Q, Jin J, et al. Wearable emotion recognition using heart rate data from a smart bracelet. Sensors. 2020; 20(3):1-19.
[Google Scholar]
[6]Liu H, Zhang Y, Li Y, Kong X. Review on emotion recognition based on electroencephalography. Frontiers in Computational Neuroscience. 2021; 15:1-15.
[Google Scholar]
[7]Guger C, Ince NF, Korostenskaja M, Allison BZ. Brain-computer interface research: a state-of-the-art summary 11. Brain-Computer Interface Research 2024 (pp. 1-11). Cham: Springer Nature Switzerland.
[Google Scholar]
[8]Kawala-sterniuk A, Browarska N, Al-bakri A, Pelc M, Zygarlicki J, Sidikova M, et al. Summary of over fifty years with brain-computer interfaces-a review. Brain Sciences. 2021; 11(1):1-41.
[Crossref] [Google Scholar]
[9]Maria MA, Akhand MA, Hossain AA, Kamal MA, Yamada K. A comparative study on prominent connectivity features for emotion recognition from EEG. IEEE Access. 2023; 11:37809-31.
[Crossref] [Google Scholar]
[10]Longo C, Shankar A, Nuttall P. “It’s not easy living a sustainable lifestyle”: how greater knowledge leads to dilemmas, tensions and paralysis. Journal of Business Ethics. 2019; 154:759-79.
[Crossref] [Google Scholar]
[11]Xu X, Zuo Y, Chen S, Hatami A, Gu H. Advancements in brain research: the in vivo/in vitro electrochemical detection of neurochemicals. Biosensors. 2024; 14(3):1-32.
[Crossref] [Google Scholar]
[12]Vansteensel MJ, Jarosiewicz B. Brain-computer interfaces for communication. Handbook of Clinical Neurology. 2020; 168:67-85.
[Crossref] [Google Scholar]
[13]Haas LF. Hans berger (1873–1941), richard caton (1842–1926), and electroencephalography. Journal of Neurology, Neurosurgery & Psychiatry. 2003; 74(1).
[Crossref] [Google Scholar]
[14]Khan AA, Laghari AA, Shaikh AA, Dootio MA, Estrela VV, Lopes RT. A blockchain security module for brain-computer interface (BCI) with multimedia life cycle framework (MLCF). Neuroscience Informatics. 2022; 2(1):100030.
[Crossref] [Google Scholar]
[15]Pandarinath C, Bensmaia SJ. The science and engineering behind sensitized brain-controlled bionic hands. Physiological Reviews. 2022; 102(2):551-604.
[Crossref] [Google Scholar]
[16]Li D, Xu J, Wang J, Fang X, Ji Y. A multi-scale fusion convolutional neural network based on attention mechanism for the visualization analysis of EEG signals decoding. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2020; 28(12):2615-26.
[Crossref] [Google Scholar]
[17]Song R, Zhang X, Chen X, Chen X, Chen X, Yang S, et al. Decoding silent speech from high-density surface electromyographic data using transformer. Biomedical Signal Processing and Control. 2023; 80:104298.
[Crossref] [Google Scholar]
[18]Mak JN, Wolpaw JR. Clinical applications of brain-computer interfaces: current state and future prospects. IEEE Reviews in Biomedical Engineering. 2009; 2:187-99.
[Crossref] [Google Scholar]
[19]Bockbrader MA, Francisco G, Lee R, Olson J, Solinsky R, Boninger ML. Brain computer interfaces in rehabilitation medicine. PM&R. 2018; 10(9): S233-43.
[Crossref] [Google Scholar]
[20]Klaes C. Invasive brain-computer interfaces and neural recordings from humans. In Handbook of Behavioral Neuroscience 2018 (pp. 527-39). Elsevier.
[Crossref] [Google Scholar]
[21]Islam MK, Rastegarnia A. Recent advances in EEG (non-invasive) based BCI applications. Frontiers in Computational Neuroscience. 2023; 17:1151852.
[Crossref] [Google Scholar]
[22]Guo Y, Nejati H, Cheung NM. Deep neural networks on graph signals for brain imaging analysis. In international conference on image processing 2017 (pp. 3295-9). IEEE.
[Crossref] [Google Scholar]
[23]Koelsch S, Fritz T, V.camon DY, Müller K, Friederici AD. Investigating emotion with music: an fMRI study. Human Brain Mapping. 2006; 27(3):239-50.
[Crossref] [Google Scholar]
[24]Al-nafjan A, Alharthi K, Kurdi H. Lightweight building of an electroencephalogram-based emotion detection system. Brain Sciences. 2020; 10(11):1-17.
[Crossref] [Google Scholar]
[25]Zhang J, Hao Y, Wen X, Zhang C, Deng H, Zhao J, et al. Subject-independent emotion recognition based on EEG frequency band features and self-adaptive graph construction. Brain Sciences. 2024; 14(3):1-19.
[Crossref] [Google Scholar]
[26]Wei Z, Li H, Ma L, Li H. Emotion recognition based on microstate analysis from temporal and spatial patterns of electroencephalogram. Frontiers in Neuroscience. 2024; 18:1355512.
[Crossref] [Google Scholar]
[27]Mutawa AM, Hassouneh A. Multimodal real-time patient emotion recognition system using facial expressions and brain EEG signals based on machine learning and log-sync methods. Biomedical Signal Processing and Control. 2024; 91:105942.
[Crossref] [Google Scholar]
[28]Wei C, Chen LL, Song ZZ, Lou XG, Li DD. EEG-based emotion recognition using simple recurrent units network and ensemble learning. Biomedical signal processing and control. 2020; 58:101756.
[Crossref] [Google Scholar]
[29]Pandey P, Seeja KR. Subject independent emotion recognition from EEG using VMD and deep learning. Journal of King Saud University-Computer and Information Sciences. 2022; 34(5):1730-8.
[Crossref] [Google Scholar]
[30]Li J, Hua H, Xu Z, Shu L, Xu X, Kuang F, et al. Cross-subject EEG emotion recognition combined with connectivity features and meta-transfer learning. Computers in Biology and Medicine. 2022; 145:105519.
[Crossref] [Google Scholar]
[31]Peng Y, Wang W, Kong W, Nie F, Lu BL, Cichocki A. Joint feature adaptation and graph adaptive label propagation for cross-subject emotion recognition from EEG signals. IEEE Transactions on Affective Computing. 2022; 13(4):1941-58.
[Crossref] [Google Scholar]
[32]Bano KS, Bhuyan P, Ray A. EEG-based brain computer interface for emotion recognition. In 5th international conference on computational intelligence and networks 2022 (pp. 1-6). IEEE.
[Crossref] [Google Scholar]
[33]Tao W, Li C, Song R, Cheng J, Liu Y, Wan F, et al. EEG-based emotion recognition via channel-wise attention and self-attention. IEEE Transactions on Affective Computing. 2020; 14(1):382-93.
[Crossref] [Google Scholar]
[34]Salankar N, Mishra P, Garg L. Emotion recognition from EEG signals using empirical mode decomposition and second-order difference plot. Biomedical Signal Processing and Control. 2021; 65:102389.
[Crossref] [Google Scholar]
[35]Wang F, Wu S, Zhang W, Xu Z, Zhang Y, Wu C, et al. Emotion recognition with convolutional neural network and EEG-based EFDMs. Neuropsychologia. 2020; 146:107506.
[Crossref] [Google Scholar]
[36]Dharia SY, Valderrama CE, Camorlinga SG. Multimodal deep learning model for subject-independent EEG-based emotion recognition. In Canadian conference on electrical and computer engineering 2023 (pp. 105-10). IEEE.
[Crossref] [Google Scholar]
[37]Farokhah L, Sarno R, Fatichah C. Cross-subject channel selection using modified relief and simplified CNN-based deep learning for EEG-based emotion recognition. IEEE Access. 2023; 11:110136-50.
[Crossref] [Google Scholar]
[38]Hwang S, Ki M, Hong K, Byun H. Subject-independent EEG-based emotion recognition using adversarial learning. In 8th international winter conference on brain-computer interface 2020 (pp. 1-4). IEEE.
[Crossref] [Google Scholar]
[39]Dhara T, Singh PK, Mahmud M. A fuzzy ensemble-based deep learning model for EEG-based emotion recognition. Cognitive Computation. 2023:1-5.
[Crossref] [Google Scholar]
[40]Keelawat P, Thammasan N, Kijsirikul B, Numao M. Subject-independent emotion recognition during music listening based on EEG using deep convolutional neural networks. In 15th international colloquium on signal processing & its applications 2019 (pp. 21-6). IEEE.
[Crossref] [Google Scholar]
[41]Kanuboyina V, Shankar T, Penmetsa RR. Electroencephalography based human emotion state classification using principal component analysis and artificial neural network. Multiagent and Grid Systems. 2022; 18(3-4):263-78.
[Crossref] [Google Scholar]
[42]Hancer E, Subasi A. EEG-based emotion recognition using dual tree complex wavelet transform and random subspace ensemble classifier. Computer Methods in Biomechanics and Biomedical Engineering. 2023; 26(14):1772-84.
[Crossref] [Google Scholar]
[43]Sharma R, Pachori RB, Sircar P. Automated emotion recognition based on higher order statistics and deep learning algorithm. Biomedical Signal Processing and Control. 2020; 58:101867.
[Crossref] [Google Scholar]
[44]Liu R, Chao Y, Ma X, Sha X, Sun L, Li S, Chang S. ERTNet: an interpretable transformer-based framework for EEG emotion recognition. Frontiers in Neuroscience. 2024; 18:1320645.
[Crossref] [Google Scholar]
[45]Li G, Yuan B, Ouyang D, Li W, Pan Y, Guo Z, et al. Emotion recognition based on selected EEG signals by common spatial pattern. IEEE Sensors Journal. 2024.
[Crossref] [Google Scholar]
[46]Liu W, Qiu JL, Zheng WL, Lu BL. Comparing recognition performance and robustness of multimodal deep learning models for multimodal emotion recognition. IEEE Transactions on Cognitive and Developmental Systems. 2021; 14(2):715-29.
[Crossref] [Google Scholar]