Early Detection of Fake News in Social Media: A Hybrid Multimodal Architecture
Keywords:
Attention mechanism, early detection, FakeNewsNet, Fake news detection, GRU, Misinformation, Multimodal learning, RoBERTa, Social media, TransformerAbstract
The rapid proliferation of fake news on social media platforms has emerged as a critical societal challenge, undermining public trust, distorting political discourse, and impacting real-world decision-making. This paper presents a comprehensive hybrid multimodal framework for the early detection of fake news, designed to operate in the initial stages of news propagation — within the first 5 to 15 minutes of a news post appearing online. The proposed architecture integrates three complementary streams of information: (1) news content features extracted using both GloVe-CNN and transformer-based (RoBERTa) models, (2) social context features derived from tweet responses and user credibility profiles, and (3) visual features from accompanying images processed through a ResNet-50 backbone. These streams are fused using a transformer-style cross-modal attention mechanism, enabling the model to capture complex inter-modal dependencies. The social context module employs a self-attention mechanism to identify discriminative tweets globally, and Gated Recurrent Units (GRUs) to model temporal dynamics locally. They evaluate the proposed framework on the FakeNewsNet dataset (PolitiFact and GossipCop subsets) as well as Fakeddit, Twitter15, and Weibo benchmarks. The proposed model achieves an accuracy of 89.5% on Fakeddit, 93.1% on Twitter15, 88.3% on Weibo, and 87.8% on FakeNewsNet — consistently outperforming existing state-of-the-art models, including Att-RNN, EANN, MVAE, and BMMFN. Ablation studies confirm the contribution of each module, and explainability analyses using Grad-CAM and SHAP validate the interpretability of the model's decisions.
References
K. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, “Fake news detection on social media: A data mining perspective,” ACM SIGKDD Explorations Newsletter, vol. 19, no. 1, pp. 22–36, Sep. 2017.
H. Allcott and M. Gentzkow, “Social media and fake news in the 2016 election,” Journal of Economic Perspectives, vol. 31, no. 2, pp. 211–236, May 2017.
M. Gupta, P. Zhao, and J. Han, “Evaluating event credibility on Twitter,” in Proceedings of the 2012 SIAM International Conference on Data Mining, Apr. 2012, pp. 153–164.
S. Afroz, M. Brennan, and R. Greenstadt, “Detecting hoaxes, frauds, and deception in writing style online,” in Proceedings of the IEEE Symposium on Security and Privacy, May 2012, pp. 461–475.
N. Capuano, G. Fenza, V. Loia, and D. F. Nota, “Content-based fake news detection with machine and deep learning: A systematic review,” Neurocomputing, vol. 530, pp. 91–103, Apr. 2023.
C. Castillo, M. Mendoza, and B. Poblete, “Information credibility on Twitter,” in Proceedings of the 20th International Conference on World Wide Web, Mar. 2011, pp. 675–684.
F. Yang, Y. Liu, X. Yu, and M. Yang, “Automatic detection of rumor on Sina Weibo,” in Proceedings of the ACM SIGKDD Workshop on Mining Data Semantics, Aug. 2012, pp. 1–7.
Zubiaga, A. Aker, K. Bontcheva, M. Liakata, and R. Procter, “Detection and resolution of rumours in social media: A survey,” ACM Computing Surveys, vol. 51, no. 2, pp. 1–36, Feb. 2018.
Z. Jin, J. Cao, H. Guo, Y. Zhang, and J. Luo, “Multimodal fusion with recurrent neural networks for rumor detection on microblogs,” in Proceedings of the 25th ACM International Conference on Multimedia, Oct. 2017, pp. 795–816.
Y. Wang, F. Ma, Z. Jin, Y. Yuan, G. Xun, K. Jha, L. Su, and J. Gao, “EANN: Event adversarial neural networks for multi-modal fake news detection,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Jul. 2018, pp. 849–857.
D. Khattar, J. S. Goud, M. Gupta, and V. Varma, “MVAE: Multimodal variational autoencoder for fake news detection,” in Proceedings of The World Wide Web Conference, May 2019, pp. 2915–2921.
N. M. Tuan and P. Q. Minh, “Multimodal fusion with BERT and attention mechanism for fake news detection,” in Proceedings of the IEEE RIVF International Conference on Computing and Communication Technologies, Aug. 2021, pp. 1–6.
K. Yu, S. Jiao, and Z. Ma, “Fake news detection based on BERT multi-domain and multi-modal fusion network,” Computer Vision and Image Understanding, vol. 252, p. 104301, Feb. 2025.
M. Jiang, C. Jing, L. Chen, Y. Wang, and S. Liu, “An application study on multimodal fake news detection based on ALBERT-ResNet50 model,” Multimedia Tools and Applications, vol. 83, no. 3, pp. 8689–8706, Jan. 2024.
K. Shu, D. Mahudeswaran, S. Wang, D. Lee, and H. Liu, “FakeNewsNet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media,” Big Data, vol. 8, no. 3, pp. 171–188, Jun. 2020.
Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision, vol. 128, no. 2, pp. 336–359, Feb. 2020.
S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Advances in Neural Information Processing Systems, vol. 30, 2017.
