AI-Based Multimedia Authenticity Detection System (MMADS) for Combating Digital Misinformation
Keywords:
CNN-based detection, Confidence score, Deepfake detection, Digital misinformation, FastAPI, Multimedia authenticity detection, Phishing detection, TensorFlowAbstract
The high rate of spreading deepfakes, AI-created images, synthetically cloned audio, phishing emails, and malicious URLs constitutes an increasing threat to the integrity of digital information, trust, and institutional security. Current detection systems are mostly unimodal, focusing on each type of content separately, and, therefore, do not offer full protection against the current multi-vector misinformation attacks. This study introduces the Multimedia Authenticity Detection System (MMADS), which is an integrated artificial intelligence system designed to identify counterfeited online content in five media types, which include pictures, videos, audio files, web addresses, and email messages. The MMADS architecture is divided into five functional layers, namely user interface, backend API, AI processing, decision output, and persistent storage, built upon a production-grade technology stack, which consists of React.js with TypeScript on the frontend, FastAPI with Uvicorn on the backend, a core detection engine based on TensorFlow and Keras CNN models, and Supabase with PostgreSQL to store results. Other libraries, such as OpenCV and Pillow to support visual preprocessing, Librosa and PyDub to extract audio features, and NumPy to perform numerical computations, are also used. Each detection model is trained on massive-scale data and, to guarantee data confidentiality and performance autonomy, the model is run in-house. In each of the media items under analysis, MMADS produces a binary decision, Authentic or Fake, and a confidence score, allowing checking the material with transparency and responsibility. Experimental analysis has shown that the system has an overall detection accuracy of 96.8 and an AUC-ROC of 0.972, and an end-to-end response latency of 142 milliseconds per media item. The main advance of this study is a platform for scalable, explainable, and computationally efficient multi-modal authenticity detection, which is compatible with the United Nations Sustainable Development Goal 16, which is digital trust, peace, and institutional integrity.
References
I. Goodfellow et al., “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, Oct. 2020.
R. Rombach, A. Blattmann, D. Lorenz, P. Esser and B. Ommer, “High-resolution image synthesis with latent diffusion models,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 10674–10685.
C. Vaccari and A. Chadwick, “Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news,” Social Media + Society, vol. 6, no. 1, Feb. 2020.
B. Dolhansky, R. Howes, B. Pflaum, N. Baram, and C. C. Ferrer, “The deepfake detection challenge (DFDC) preview dataset,” arXiv, Oct. 2019.
H. Farid, “Image forgery detection,” IEEE Signal Processing Magazine, vol. 26, no. 2, pp. 16–25, Mar. 2009.
J. H. Bappy, A. K. Roy-Chowdhury, J. Bunk, L. Nataraj and B. S. Manjunath, “Exploiting spatial structure for localizing manipulated image regions,” 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 4980–4989.
A. Rössler, D. Cozzolino, L. Verdoliva, C. Rieß, J. Thies, and M. Nießner, “FaceForensics++: Learning to detect manipulated facial images,” arXiv, Jan. 2019.
F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 1800–1807.
D. Güera and E. J. Delp, “Deepfake video detection using recurrent neural networks,” 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Auckland, New Zealand, 2018, pp. 1–6.
Y. Li, M. -C. Chang and S. Lyu, “In Ictu Oculi: Exposing AI created fake videos by detecting eye blinking,” 2018 IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong, China, 2018, pp. 1–7.
M. Todisco et al., “ASVspoof 2019: Future horizons in spoofed and fake audio detection,” Proceedings of Interspeech 2019, 2019, pp. 1008–1012.
J. Yi, C. Wang, J. Tao, X. Zhang, C. Y. Zhang, and Y. Zhao, “Audio deepfake detection: A survey,” arXiv, Aug. 2023.
Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv, Jul. 2019.
R. Zellers et al., “Defending against neural fake news,” Proceedings of the 33rd International Conference on Neural Information Processing Systems, May 2019, pp. 9054–9065.
I. Amerini, L. Galteri, R. Caldelli and A. Del Bimbo, “Deepfake video detection through optical flow based CNN,” 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019, pp. 1205–1207.
R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-CAM: Visual explanations from deep networks via gradient-based localization,” International Journal of Computer Vision, vol. 128, pp. 336–359, Feb. 2020.
R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega-Garcia, “DeepFakes and beyond: A survey of face manipulation and fake detection,” Information Fusion, vol. 64, pp. 131–148, Dec. 2020.
M. E. Kaminski and G. Malgieri, “Algorithmic impact assessments under the GDPR: Producing multi-layered explanations,” International Data Privacy Law, vol. 11, no. 2, pp. 125–144, Apr. 2021.
D. Afchar, V. Nozick, J. Yamagishi and I. Echizen, “MesoNet: A compact facial video forgery detection network,” 2018 IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong, China, 2018, pp. 1–7.
A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar and L. Fei-Fei, “Large-scale video classification with convolutional neural networks,” 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 1725–1732.
