(Publisher of Peer Reviewed Open Access Journals)

International Journal of Advanced Technology and Engineering Exploration (IJATEE)

ISSN (Print):2394-5443    ISSN (Online):2394-7454
Volume-11 Issue-114 May-2024
Full-Text PDF
Paper Title : Multimodal fake news detection using hyperparameter-tuned BERT and ResNet110
Author Name : Chetan Agrawal, Anjana Pandey and Sachin Goyal
Abstract :

Globally, the usage of social media has significantly increased and has become the most common way for people to deplete news. The easy sharing of multimedia content on social media has caused the fake news dimension, which threatens the stability as well as security of the society. Fake news detection (FND) in social media becomes challenging, because of which various tools are developed to detect them. Multimodal FND aims to determine fake data by text as well as images. Most commonly, researchers identify fake news only as text, but not as images. This research proposes a multimodal method for the detection and classification of fake news into real or fake. The proposed multimodal-based convolutional neural network (CNN) combines the designs of both text and image of fake news. This method utilizes two classification methods named hyperparameter tuning based bidirectional encoder representations from transformers (HTBERT) for text, and ResNet110 for images. Fakeddit dataset has used to estimate and evaluate the performance. The experimental results of the proposed ResNet110+HTBERT model achieves respective accuracy, precision, recall and F1-score values of about 0.931, 0.944, 0.942, and 0.946, which is superior when compared to the existing methods, recurrent CNN (RCNN) and fine-grained multimodal fusion network (FMFN). From the analysis, it is evident that the proposed method ResNet110+HTBERT achieves an accuracy of 0.931, and hence shows better results for overall metrics when compared to the existing methods of RCNN and FMFN.

Keywords : Convolutional neural network, Hyperparameter tuning based bidirectional encoder representations from transformers, Multimodal fake news detection, ResNet110, Social media.
Cite this article : Agrawal C, Pandey A, Goyal S. Multimodal fake news detection using hyperparameter-tuned BERT and ResNet110. International Journal of Advanced Technology and Engineering Exploration. 2024; 11(114):759-772. DOI:10.19101/IJATEE.2023.10102314.
References :
[1]Katariya P, Gupta V, Arora R, Kumar A, Dhingra S, Xin Q, et al. A deep neural network-based approach for fake news detection in regional language. International Journal of Web Information Systems. 2022; 18(5/6):286-309.
[Crossref] [Google Scholar]
[2]Fu L, Peng H, Liu S. KG-MFEND: an efficient knowledge graph-based model for multi-domain fake news detection. The Journal of Supercomputing. 2023; 79(16):18417-44.
[Crossref] [Google Scholar]
[3]Dixit DK, Bhagat A, Dangi D. An accurate fake news detection approach based on a levy flight honey badger optimized convolutional neural network model. Concurrency and Computation: Practice and Experience. 2023; 35(1):e7382.
[Crossref] [Google Scholar]
[4]Katarya R, Dahiya D, Checker S. Fake news detection system using featured-based optimized msvm classification. IEEE Access. 2022; 10:113184-99.
[Crossref] [Google Scholar]
[5]Lai CM, Chen MH, Kristiani E, Verma VK, Yang CT. Fake news classification based on content level features. Applied Sciences. 2022; 12(3):1-21.
[Crossref] [Google Scholar]
[6]Liao Q, Chai H, Han H, Zhang X, Wang X, Xia W, et al. An integrated multi-task model for fake news detection. IEEE Transactions on Knowledge and Data Engineering. 2021; 34(11):5154-65.
[Crossref] [Google Scholar]
[7]Meel P, Vishwakarma DK. HAN, image captioning, and forensics ensemble multimodal fake news detection. Information Sciences. 2021; 567:23-41.
[Crossref] [Google Scholar]
[8]Singhal S, Pandey T, Mrig S, Shah RR, Kumaraguru P. Leveraging intra and inter modality relationship for multimodal fake news detection. In companion proceedings of the web conference 2022 (pp. 726-34). ACM.
[Crossref] [Google Scholar]
[9]Papadopoulos SI, Koutlis C, Papadopoulos S, Petrantonakis PC. VERITE: a robust benchmark for multimodal misinformation detection accounting for unimodal bias. International Journal of Multimedia Information Retrieval. 2024; 13(1):1-15.
[Crossref] [Google Scholar]
[10]Raza S, Ding C. Fake news detection based on news content and social contexts: a transformer-based approach. International Journal of Data Science and Analytics. 2022; 13(4):335-62.
[Crossref] [Google Scholar]
[11]Cui X, Yang L. Fake news detection in social media based on multi-modal multi-task learning. International Journal of Advanced Computer Science and Applications. 2022; 13(7):1-7.
[Crossref] [Google Scholar]
[12]Dong D, Lin F, Li G, Liu B. Similarity-aware attention network for multimodal fake news detection. Third international conference on big data & artificial intelligence & software engineering 2022 (pp.76-85).
[Google Scholar]
[13]Ying Q, Hu X, Zhou Y, Qian Z, Zeng D, Ge S. Bootstrapping multi-view representations for fake news detection. In proceedings of the AAAI conference on artificial intelligence 2023 (pp. 5384-92).
[Crossref] [Google Scholar]
[14]Hakak S, Alazab M, Khan S, Gadekallu TR, Maddikunta PK, Khan WZ. An ensemble machine learning approach through effective feature extraction to classify fake news. Future Generation Computer Systems. 2021; 117:47-58.
[Crossref] [Google Scholar]
[15]Meel P, Vishwakarma DK. A temporal ensembling based semi-supervised ConvNet for the detection of fake news articles. Expert Systems with Applications. 2021; 177:115002.
[Crossref] [Google Scholar]
[16]Mughaid A, Al-zu’bi S, Al AA, Al-amrat R, Alajmi R, Zitar RA, et al. An intelligent cybersecurity system for detecting fake news in social media websites. Soft Computing. 2022; 26(12):5577-91.
[Crossref] [Google Scholar]
[17]Zhou Y, Yang Y, Ying Q, Qian Z, Zhang X. Multi-modal fake news detection on social media via multi-grained information fusion. In proceedings of the international conference on multimedia retrieval 2023 (pp. 343-52). ACM.
[Crossref] [Google Scholar]
[18]Raj C, Meel P. ConvNet frameworks for multi-modal fake news detection. Applied Intelligence. 2021; 51(11):8132-48.
[Crossref] [Google Scholar]
[19]Kaliyar RK, Goswami A, Narang P. EchoFakeD: improving fake news detection in social media with an efficient deep neural network. Neural Computing and Applications. 2021; 33:8597-613.
[Crossref] [Google Scholar]
[20]Song C, Ning N, Zhang Y, Wu B. A multimodal fake news detection model based on crossmodal attention residual and multichannel convolutional neural networks. Information Processing & Management. 2021; 58(1):102437.
[Crossref] [Google Scholar]
[21]Liu P, Qian W, Xu D, Ren B, Cao J. Multi-modal fake news detection via bridging the gap between Modals. Entropy. 2023; 25(4):1-17.
[Crossref] [Google Scholar]
[22]Segura-bedmar I, Alonso-bartolome S. Multimodal fake news detection. Information. 2022; 13(6):1-16.
[Crossref] [Google Scholar]
[23]Ying L, Yu H, Wang J, Ji Y, Qian S. Multi-level multi-modal cross-attention network for fake news detection. IEEE Access. 2021; 9:132363-73.
[Crossref] [Google Scholar]
[24]Guo Y, Song W. A temporal-and-spatial flow based multimodal fake news detection by pooling and attention blocks. IEEE Access. 2022; 10:131498-508.
[Crossref] [Google Scholar]
[25]Wang J, Mao H, Li H. FMFN: fine-grained multimodal fusion networks for fake news detection. Applied Sciences. 2022; 12(3):1-12.
[Crossref] [Google Scholar]
[26]Jing J, Wu H, Sun J, Fang X, Zhang H. Multimodal fake news detection via progressive fusion networks. Information Processing & Management. 2023; 60(1):103120.
[Crossref] [Google Scholar]
[27]Kumari R, Ekbal A. Amfb: attention based multimodal factorized bilinear pooling for multimodal fake news detection. Expert Systems with Applications. 2021; 184:115412.
[Crossref] [Google Scholar]
[28]Rai N, Kumar D, Kaushik N, Raj C, Ali A. Fake news classification using transformer based enhanced LSTM and BERT. International Journal of Cognitive Computing in Engineering. 2022; 3:98-105.
[Crossref] [Google Scholar]
[29]Li S, Yao T, Li S, Yan L. Semantic‐enhanced multimodal fusion network for fake news detection. International Journal of Intelligent Systems. 2022; 37(12):12235-51.
[Crossref] [Google Scholar]
[30]Mehta D, Dwivedi A, Patra A, Anand KM. A transformer-based architecture for fake news classification. Social Network Analysis and Mining. 2021; 11:1-2.
[Crossref] [Google Scholar]
[31]Palani B, Elango S, Viswanathan KV. CB-fake: a multimodal deep learning framework for automatic fake news detection using capsule neural network and BERT. Multimedia Tools and Applications. 2022; 81(4):5587-620.
[Crossref] [Google Scholar]
[32]Kaliyar RK, Goswami A, Narang P. FakeBERT: fake news detection in social media with a BERT-based deep learning approach. Multimedia Tools and Applications. 2021; 80(8):11765-88.
[Google Scholar]
[33]Aslam N, Ullah KI, Alotaibi FS, Aldaej LA, Aldubaikil AK. Fake detect: a deep learning ensemble model for fake news detection. Complexity. 2021; 2021:1-8.
[Crossref] [Google Scholar]
[34]Choudhary A, Arora A. Linguistic feature based learning model for fake news detection and classification. Expert Systems with Applications. 2021; 169:114171.
[Crossref] [Google Scholar]
[35]Xue J, Wang Y, Tian Y, Li Y, Shi L, Wei L. Detecting fake news by exploring the consistency of multimodal data. Information Processing & Management. 2021; 58(5):102610.
[Crossref] [Google Scholar]
[36]Guo Y. A mutual attention based multimodal fusion for fake news detection on social network. Applied Intelligence. 2023; 53(12):15311-20.
[Crossref] [Google Scholar]
[37]Yang P, Ma J, Liu Y, Liu M. Multi-modal transformer for fake news detection. Mathematical Biosciences and Engineering: MBE. 2023; 20(8):14699-717.
[Google Scholar]
[38]Sastrawan IK, Bayupati IP, Arsa DM. Detection of fake news using deep learning CNN–RNN based methods. ICT Express. 2022; 8(3):396-408.
[Crossref] [Google Scholar]
[39]Al OA, Khotanlou H, Mansoorizadeh M, Zabihzadeh D. Multimodal fake-news recognition using ensemble of deep learners. Entropy. 2022; 24(9):1-16.
[Crossref] [Google Scholar]
[40]Uppada SK, Patel P. An image and text-based multimodal model for detecting fake news in OSN’s. Journal of Intelligent Information Systems. 2023; 61(2):367-93.
[Crossref] [Google Scholar]
[41]https://paperswithcodecom/dataset/fakeddit.. Accessed 23 April 2024.
[42]Keya AJ, Wadud MA, Mridha MF, Alatiyyah M, Hamid MA. AugFake-BERT: handling imbalance through augmentation of fake news using BERT to enhance the performance of fake news classification. Applied Sciences. 2022; 12(17):1-21.
[Crossref] [Google Scholar]
[43]Shishah W. Fake news detection using BERT model with joint learning. Arabian Journal for Science and Engineering. 2021; 46(9):9115-27.
[Crossref] [Google Scholar]
[44]Liu Y, Pu H, Sun DW. Efficient extraction of deep image features using convolutional neural network (CNN) for applications in detecting and analysing complex food matrices. Trends in Food Science & Technology. 2021; 113:193-204.
[Google Scholar]
[45]Barbhuiya AA, Karsh RK, Jain R. CNN based feature extraction and classification for sign language. Multimedia Tools and Applications. 2021; 80(2):3051-69.
[Crossref] [Google Scholar]
[46]Guo N, Gu K, Qiao J, Bi J. Improved deep CNNs based on nonlinear hybrid attention module for image classification. Neural Networks. 2021; 140:158-66.
[Crossref] [Google Scholar]
[47]Zhao W, Zhu L, Wang M, Zhang X, Zhang J. WTL-CNN: a news text classification method of convolutional neural network based on weighted word embedding. Connection Science. 2022; 34(1):2291-312.
[Crossref] [Google Scholar]
[48]Zhang F. A hybrid structured deep neural network with Word2Vec for construction accident causes classification. International Journal of Construction Management. 2022; 22(6):1120-40.
[Crossref] [Google Scholar]
[49]Zhang Y, Shi T, Lu ZM. Image splicing detection scheme based on error level analysis and local binary pattern. Journal of Network Intelligence. 2021; 6:303-12.
[Google Scholar]
[50]Singh R, Singh S. Text similarity measures in news articles by vector space model using NLP. Journal of The Institution of Engineers (India): Series B. 2021; 102:329-38.
[Crossref] [Google Scholar]