AI-generated Deepfakes in the Age of Misinformation: A Review of Methods, Impacts, and Defenses
Keywords:
Artificial intelligence, Deepfakes, Disinformation, Generative adversarial networks, MisinformationAbstract
The presence of artificial intelligence technologies, particularly those that enable the creation of hyper-realistic synthetic media, also known as deepfakes, has introduced serious challenges to information integrity. AI systems utilize deep learning techniques like generative adversarial networks (GANs) to generate sound, video, or images that would be capable of convincingly imitating real people and events. Although these technologies can potentially be applied to benefit entertainment, education, and accessibility, they are extremely dangerous when used inappropriately. Deepfakes can be used to produce misinformation, accidentally disseminate false information, and disinformation, intentionally misleading information to sway opinion or ruin reputations. The potential deepfakes hold in undermining media credibility, fueling political and social tensions, and enabling fraud or manipulation of identity is a reason why effective detection systems, ethics, and legislation need to be in place. This is an overview of the dual-use character of deepfake technology, technical foundations, its social impact, and existing mitigation and regulation.
References
H. Murphy, “The rising threat to democracy of AI-powered disinformation,” Financial Times, Jan. 2024. Available: https://www.ft.com/content/16f23c01-fa51-408e-acf5-0d30a5a1ebf2
K. Haan, R. Watts, “Over 75% of consumers are concerned about misinformation from artificial intelligence,” Forbes, Jul. 20, 2023, Available: https://www.forbes.com/advisor/business/
artificial-intelligence-consumer-sentiment/
The Economist, “Disinformation is on the rise. How does it work?” The Economist, May 1, 2024. Available: https://www.economist.com/science-and-technology/2024/05/01/disinformation-is-on-the-rise-how-does-it-work
R. Satter and N. Vasilyeva, “Insiders: Russia troll farm even zanier than indictment says,” AP News, Feb. 20, 2018. Available: https://apnews.com/article/c737b8bd4d24487db64194d95ec4d
f0
N. Kshetri, “Disinformation and misinformation in the age of artificial intelligence and the metaverse,” Computer, vol. 57, no. 12, pp. 110–116, Dec. 2024, doi: https://doi.org/10.1109/mc.2024.3461325
S. J. Nightingale and H. Farid, “AI-synthesized faces are indistinguishable from real faces and more trustworthy,” Proceedings of the National Academy of Sciences, vol. 119, no. 8, Feb. 2022, doi: https://doi.org/10.1073/pnas.2120481119
L. B. Rosenberg, “Evil twins and digital elves: How the metaverse will create new forms of fraud and deception,” Big Think, Apr. 25, 2022. Available: https://bigthink.com/the-future/metaverse-fraud-digital-twins/
R. Waltzman, “Facebook misinformation is bad enough. The metaverse will be worse,” RAND, Aug. 22, 2022, Available: https://www.rand.org/pubs/commentary/2022/08/facebook-misinformation-is-bad-enough-the-metaverse.html
Washington State Department of Financial Institutions, “Investment scams in the metaverse: How to protect yourself from losing real money in the virtual world,” Washington State Department of Financial Institutions, Sep. 1, 2022, Available: https://dfi.wa.gov/consumer/alerts/investment-scams-metaverse
G. Vani, R. Naveenkumar, R. Singha, R. Sarkar, N. Kumar, “Advancing predictive data analytics in IoT and AI leveraging real-time data for proactive operations and system resilience,” Nanotechnology Perceptions, vol. 20, no. S16, pp. 568–582, Dec. 2024, doi: https://doi.org/10.62441/nano-ntp.vi.3968
M. Hameleers, “Disinformation as a context-bound phenomenon: Toward a conceptual clarification integrating actors, intentions and techniques of creation and dissemination,” Communication Theory, vol. 33, no. 1, pp. 1–10, Feb. 2023, doi: https://doi.org/10.1093/ct/qtac021
C. D. Ruiz, “Disinformation on digital media platforms: A market-shaping approach,” New Media & Society, vol. 27, no. 4, Oct. 2023, doi: https://doi.org/10.1177/14614448231207644
G. Di Domenico, D. Nunan, and V. Pitardi, “Marketplaces of misinformation: A study of how vaccine misinformation is legitimized on social media,” Journal of Public Policy & Marketing, vol. 41, no. 4, pp. 319–335, 2022, doi: https://psycnet.apa.org/doi/10.1177/07439156221103860
