Abstract

Heart diseases continue to be the leading cause of mortality, emphasising the urgent need for advanced and accurate diagnostic systems. In this work, we present an integrated framework that synergistically integrates deep learning and reinforcement learning (RL) for the classification of cardiovascular diseases (CVDs) using multimodal physiological signals, specifically electrocardiograms (ECGs) and phonocardiograms (PCGs). The proposed architecture leverages Convolutional neural networks (CNNs) to extract spatial features and recurrent neural networks (RNNs) to capture temporal dynamics. At the same time, RL is employed to optimise decision-making and adaptively improve classification accuracy. To effectively fuse multimodal data, we implement and evaluate early, late, and hybrid fusion strategies. The framework is validated on the benchmark EPHNOGRAM dataset, achieving a peak detection accuracy of 94%. Comparative experiments demonstrate that our hybrid LSTM-BiLSTM model outperforms existing state-of-the-art techniques. These results underscore the potential of the proposed framework as a robust and intelligent system for automated CVD diagnosis, paving the way for future improvements in clinical decision support powered by deep learning and RL.

Keywords

Deep learning, Reinforcement learning, Multimodal fusion, Heart disease, and Machine learning.

References

[1] Shabani N, Saffar SS, Bavafa-valenlia K, Esmaeilzadeh N, Mohammadi-bajgriran M, Namazibaygi Z, et al. The association between age at menarche (AAM) and cardiovascular disease (CVD) risk factors in the MASHAD cohort study. Scientific Reports. 2025; 15(1):3518.

[2] Aabdalla ID, Vasumathi D. Wavelet scattering transform for ECG cardiovascular disease classification. International Journal of Artificial Intelligence and Applications. 2024; 15(1):99-111

[3] Nikhar S, Karandikar AM. Prediction of heart disease using machine learning algorithms. International Journal of Advanced Engineering, Management and Science. 2016; 2(6):239484.

[4] Ketu S, Mishra PK. An intelligent hybrid classification model for heart disease detection using imbalanced electrocardiogram signals. The Journal of Supercomputing. 2024; 80(3):4286-308.

[5] Ameen A, Fattoh IE, Abd ET, Ahmed K. Advances in ECG and PCG-based cardiovascular disease classification: a review of deep learning and machine learning methods. Journal of Big Data. 2024; 11(1):159.

[6] Nkereuwem EE, Ansa GO, Umoh UA, Essien UD, Nkereuwem EE, Asuquo MP. Machine learning based system for early heart disease detection and classification using audio signal processing approach. Global Journal of Engineering and Technology Advances. 2024; 21(1):87-104.

[7] Warnecke JM, Lasenby J, Deserno TM. Robust in-vehicle heartbeat detection using multimodal signal fusion. Scientific Reports. 2023; 13(1):20864.

[8] Zubair M, Woo S, Lim S, Kim D. Deep representation learning with sample generation and augmented attention module for imbalanced ECG classification. IEEE Journal of Biomedical and Health Informatics. 2023; 28(5):2461-72.

[9] Zhu JY, Liu H, Liu XW. Cardiovascular disease detection based on multi-modal data fusion and multi-branch residual network. In proceedings of the international conference on frontiers of artificial intelligence and machine learning 2023 (pp. 25-8). ACM.

[10] Duan J, Xiong J, Li Y, Ding W. Deep learning based multimodal biomedical data fusion: an overview and comparative review. Information Fusion. 2024:102536.

[11] Daduvy A, Kaza VS, Shuaib M, Mohzary M, Alam S, Sheneamer A. A multi-scale convolutional LSTM-dense network for robust cardiac arrhythmia classification from ECG signals. Computers in Biology and Medicine. 2025; 191:110121.

[12] Huda N, Windiarti IS. Reinforcement learning and meta-learning perspectives frameworks for future medical imaging. Bulletin of Social Informatics Theory and Application. 2024; 8(2):271-9.

[13] Ly A, Dazeley R, Vamplew P, Cruz F, Aryal S. Elastic step DQN: a novel multi-step algorithm to alleviate overestimation in deep Q-Networks. Neurocomputing. 2024; 576:127170.

[14] Saad MM, O’reilly R, Rehmani MH. A survey on training challenges in generative adversarial networks for biomedical image analysis. Artificial Intelligence Review. 2024; 57(2):19.

[15] Serhani MA, Ismail H, El-kassabi HT, Al BH. Enhancing arrhythmia prediction through an adaptive deep reinforcement learning framework for ECG signal analysis. Biomedical Signal Processing and Control. 2025; 101:107155.

[16] Shiwlani A, Ahmad A, Umar M, Dharejo N, Tahir A, Shiwlani S. Analysis of multi-modal data through deep learning techniques to diagnose CVDs: a review. International Journal of Membrane Science and Technology. 2024; 11(1):402-20.

[17] Chen Z, Yang D, Cui T, Li D, Liu H, Yang Y, et al. A novel imbalanced dataset mitigation method and ECG classification model based on combined 1D_CBAM-autoencoder and lightweight CNN model. Biomedical Signal Processing and Control. 2024; 87:105437.

[18] Prasanna KS, Challa NP, Nagaraju J. Heart disease prediction using reinforcement learning technique. In third international conference on advances in electrical, computing, communication and sustainable technologies 2023 (pp. 1-5). IEEE.

[19] Yang J, El-bouri R, O’donoghue O, Lachapelle AS, Soltan AA, Eyre DW, et al. Deep reinforcement learning for multi-class imbalanced training: applications in healthcare. Machine Learning. 2024; 113(5):2655-74.

[20] Li Y, Daho ME, Conze PH, Zeghlache R, Le BH, Tadayoni R, et al. A review of deep learning-based information fusion techniques for multimodal medical image classification. Computers in Biology and Medicine. 2024: 108635.

[21] Popovska-slavova E, Georgieva-tsaneva G. In-depth review of neural network architectures for forecasting heart rate time series data. Science Series-Innovative STEM Education. 2024; 6:203-24.

[22] Hafiz AM, Hassaballah M, Alqahtani A, Alsubai S, Hameed MA. Reinforcement learning with an ensemble of binary action deep Q-networks. Computer Systems Science & Engineering. 2023; 46(3).

[23] Hu M, Zhang J, Matkovic L, Liu T, Yang X. Reinforcement learning in medical image analysis: concepts, applications, challenges, and future directions. Journal of Applied Clinical Medical Physics. 2023; 24(2):e13898.

[24] Li QY, Tang BH, Wu YE, Yao BF, Zhang W, Zheng Y, et al. Machine learning: a new approach for dose individualization. Clinical Pharmacology & Therapeutics. 2024; 115(4):727-44.

[25] Kayikci S, Khoshgoftaar TM. Blockchain meets machine learning: a survey. Journal of Big Data. 2024; 11(1):1-29.

[26] Parthasarathy V, Santhosh R. Dense-BiGRU: densely connected bi-directional gated recurrent unit based heart failure detection using ECG signal. Journal of Cybersecurity & Information Management. 2024; 14(2).

[27] Sun C, Liu C, Wang X, Liu Y, Zhao S. Coronary artery disease detection based on a novel multi-modal deep-coding method using ECG and PCG signals. Sensors. 2024; 24(21):1-21.

[28] Alsekait DM, Shdefat AY, Nabil A, Nawaz A, Rana MR, Ahmed Z, et al. Heart-Net: a multi-modal deep learning approach for diagnosing cardiovascular diseases. Computers, Materials & Continua. 2024; 80(3):3967-90.

[29] Alasmari S, Alghamdi R, Tejani GG, Sharma SK, Mousavirad SJ. Federated learning-based multimodal approach for early detection and personalized care in cardiac disease. Frontiers in Physiology. 2025; 16:1563185.

[30] Li X, Li C, Wei Y, Sun Y, Wei J, Li X, et al. Bat: beat-aligned transformer for electrocardiogram classification. In international conference on data mining (ICDM) 2021 (pp. 320-9). IEEE.

[31] Hu Y, Chen J, Hu L, Li D, Yan J, Ying H, et al. Personalized heart disease detection via ECG digital twin generation. In proceedings of the thirty-third international joint conference on artificial intelligence 2024 (pp. 5872-81).

[32] Dhaka P, Sehrawat R, Bhutani P. An innovative approach to cardiovascular disease prediction: a hybrid deep learning model. Engineering, Technology & Applied Science Research. 2023; 13(6):12396-403.

[33] García-ordás MT, Bayón-gutiérrez M, Benavides C, Aveleira-mata J, Benítez-andrades JA. Heart disease risk prediction using deep learning techniques with feature augmentation. Multimedia Tools and Applications. 2023; 82(20):31759-73.

[34] Pandey V, Lilhore UK, Walia R. Advanced heart disease prediction through spatial and temporal feature learning with SCN-deep BiLSTM. International Journal of Computational Intelligence Systems. 2025; 18(1):25.

[35] Rani P, Kumar R, Jain A, Lamba R, Sachdeva RK, Kumar K, et al. An extensive review of machine learning and deep learning techniques on heart disease classification and prediction. Archives of Computational Methods in Engineering. 2024; 31(6):3331-49.

[36] Zaidi SA, Ghafoor A, Kim J, Abbas Z, Lee SW. Heart EnsembleNet: an innovative hybrid ensemble learning approach for cardiovascular risk prediction. Healthcare. 2025; 13(5):507.

[37] Dedeturk BA, Dedeturk BK, Bakir-gungor B. CSA-DE-LR: enhancing cardiovascular disease diagnosis with a novel hybrid machine learning approach. Peer J Computer Science. 2024; 10:e2197.

[38] Wang J, Zang J, An Q, Wang H, Zhang Z. A pooling convolution model for multi-classification of ECG and PCG signals. Computer Methods in Biomechanics and Biomedical Engineering. 2025; 28(5):628-41.

[39] Zhou C, Dai P, Hou A, Zhang Z, Liu L, Li A, et al. A comprehensive review of deep learning-based models for heart disease prediction. Artificial Intelligence Review. 2024; 57(10):263.

[40] Kazemnejad A, Gordany P, Sameni R. Ephnogram: a simultaneous electrocardiogram and phonocardiogram database. PhysioNet. 2021; 101 (23):215–20.

Cite this article