The Era of Artificial Intelligence: How People Distinguish the Authenticity of Information

Foreword

“AI fake news is common in our life.”

With the rapid development of artificial intelligence in today’s era and its gradual penetration into our daily life, it becomes increasingly challenging to assess the credibility of any information. Nowadays, we have weakened the evidence value of images, videos, and even sounds. With the easy access to information and algorithms to produce any kind of society news, the dividing line between the true information and the false becomes blurred, and information security is undermined.

The recent case of the shooting at Bondi Beach as an example could explain AI fake news. A so-called “scene photo” on social networks quickly attracted the attention of tens of thousands of people and became widely discussed. This particular picture was so well-composed and detailed that it had a typical news report visual style that it was difficult to evaluate with accuracy in such an emergency situation. Later, it was revealed to the world that the information depicted in this photograph was fake content generated by artificial intelligence. However, before this disclosure was made, the photo was already circulated among thousands of people.

https://rmtzx.sciencenet.cn/kxwsprint/65826e20e4b03b5da6d05977.png

Therefore, in the context where all kinds of images, sounds, and narrations may be generated using artificial intelligence, how should we distinguish the truth and fake ? In the time of authoritative releases being discredited by the general public and AI technology, who should be trusted? Furthermore, are people capable of distinguishing the truth anymore?

Current situation: AI false content has permeated every area

Responses from ChatGPT, Copilot, Gemini and Perplexity were studied in the research.

Composite: Rex/Shutterstock/Getty Images

At present, AI-generated false information is no longer a sporadic occurrence but is gradually spreading across various aspects of our everyday life. Some commonly used AI-generated products include fake gaming guides on game forums, even though these guides are usually complete and structured, but may sometimes distort users’ judgments in detail. Furthermore, in medical and public spheres, the AI generated fake news and so-called “popular science content” can affect the public perception by distorting information and appealing to experts’ professional voice and style (park, 2024).

However, the most worrying aspect of AI disinformation is that it may not only create new information but become a simple replica of a human being during data simulation in order to create disinformation. AI has became a simple human like intelligent model in the process of massive data simulation (Crawford, 2021) Thus, according to the recent research, Apple AI has been found pushing fake BBC news by using the typical layout style and voice associated with the BBC. What is more, additional studies found out that more than half of the information generated by multiple AI systems, such as ChatGPT, Microsoft Copilot, and Google Gemini, contained significant inaccuracies (according to the BBC). Furthermore, in specialized spheres of human activity, including sciences, AI literature is gaining popularity.

Why disinformation can continue to spread despite multiple regulatory mechanisms

Given all the information above, why does disinformation still spread on such a massive scale despite the existence of various regulatory measures? The emergence of artificial intelligence has essentially altered the way of information production and dissemination. Firstly, generative AI technologies decrease the threshold of information generation, thus making it independent of any professional institutions and media organizations. As a result, the content may be quickly generated by both users, businesses, and anonymous profiles and will contain the same narrative structure and completeness of any professional media. In this regard, the creation of disinformation stops being an individual’s action and turns into a decentralized, low-cost, and high-frequency process(Milosev& Ackovska , 2011).

Secondly, the high realism of AI-generated content makes the task of its recognition even more complicated. As far as language, style, format of data presentation, and visual design of AI content goes, its imitation of the professional content of the news, expert opinion, and even the speech of authoritative institutions is so accurate that distinguishing fake from true is becoming increasingly difficult. This “imitation” of the appearance of any piece of information makes the recipient rely on appearance, rather than investigate in depth the origins of information received, which considerably simplifies the process of content judgment.

In addition, the complexity of the process of information production makes the information governance task even more complicated. AI-generated content requires a huge volume of data and sophisticated algorithms to operate, and the exact information pathway is often not entirely transparent. Additionally, the “black box” nature of AI algorithms results in the inability to recognize the author of this content (Pasquale, 2015). Consequently, as a result, if AI disinformation does not contain any author’s information, existing systems for tracing and regulation fail to cope with the issue. Moreover, artificial intelligence can operate in mass production mode, creating large amounts of content in an extremely short period of time, thus overwhelming any human review and analysis mechanisms. This results in regulatory mechanisms being incapable of coping with AI-generated information.

Finally, the existing legal and institutional approaches are also insufficient, due to the emergence of AI-based information generation and transmission processes. In particular, many countries’ legal frameworks do not cover such new technologies and leave space for ambiguity in regulating them. In practice, however, the issues caused by AI are often addressed by platform regulations and content management mechanisms, which leads to a clear institutional gap in governance.

When falsehoods and trust are connected, where should the public turn to do

AI fake news – 搜索 图片

When false information and public trust mechanisms come into contact, the problem stops being merely technical and transforms into a more complex societal dilemma. Firstly, in terms of informational environment, AI disinformation features a high level of automation and scaling. Massive amounts of generated content complicate the process of individual judgment as there is not enough time and opportunity to investigate every single content item, and the spread of numerous distorted facts leads to the emergence of secondary distortion and overburdens valuable content.

https://th.bing.com/th/id/OIP.3KRsKdOfrj9kPvizjygvpwHaEK?w=279&h=180&c=7&r=0&o=5&dpr=1.3&pid=1.7

Actually, information judgment principles become altered as well. As far as an individual’s perception is concerned, as the informational environment becomes complicated, people tend to resort to emotional perception rather than logical. This shift is defined as “post-truth,” that is, the preference for information is based not on whether it is correct but on whether it evokes emotional reaction from its receiver. This tendency to shift from logical to emotional perception facilitates the emergence and spread of contradicting, inflammatory, and even explanatory content.

Secondly, in terms of social and institutional mechanisms of information dissemination, the very trust structure is transformed as well. On one hand, institutions start applying AI-based technologies to produce information, thus obscuring the origins of content; On the other hand, AI content can easily imitate the stylistic and linguistic features of professional content, including that of authoritative media outlets. As a result of these two tendencies’ convergence, any institution loses its unique nature, and the traditional, institution-centric trust structure undergoes a significant change – namely, it is being deconstructed (Botha & Pieterse, 2020). In this case, the result includes public distrust towards media and expert content, deterioration of discussions’ quality, and even distrust of the whole notion of information and artificial intelligence technologies themselves.

Overall, the impact of AI disinformation is not limited to the content itself but affects the very foundation of the information exchange system by altering the information environment, cognitive processes, and trust structures of societies. As a result, instead of being faced with a dilemma of whether to trust the information or not, modern society has to find the answer to the question of how to preserve its ability to rationally judge and trust information in the environment where falsehoods are being recreated.

Case study: Fake photographs of Bondi Beach attract tens of millions of retweets

Misinformation spreads online after the Bondi Beach terror attack. (ABC NEWS Verify)

In the Bondi Beach shooting case, the process of AI disinformation spread is especially evident. Arsen Ostrovsky, one of the victims of this tragedy, posted pictures on social media featuring him, his clothes stained with blood. In this case, the photograph was initially created as a mere recording of the tragic event. However, this original photo with real origin was then replicated by somebody else and using AI technology created new variants of it and distributed it among internet users. New iterations of the picture gained additional appeal compared to the original one and were supplemented with new stories that gave the image a racial, religious, and political implication. Hence, AI does not create illusions, which are not related to the reality at all but works by recreating, distorting, and giving new interpretations to something real, thus trying to make the false information sound as close to the truth as possible.

In this way, one of the key aspects of AI disinformation in contemporary society is revealed. Rather than opposing truth to the false, it allows them to be combined together via technological means. While original photograph adds initial credibility, the subsequent modification of it by AI technology makes this photograph emotionally oriented and gives it a certain narrative frame, thus turning it from a mere recording of an event into its “meaning-shaping”. In this way, the most dangerous thing about AI false content is not in its fabrication but in the combination of the latter with elements of reality that make the false content more contagious and appealing to the masses.

This effect is especially pronounced in the case of suddenly crisis situations as crises involve heightened levels of uncertainty, emotional intensity, and information deficits. Under this circumstances, the desire to receive an explanation of events and their causes becomes higher than the desire to calmly wait for verification. At this point, information with high visual appeal and clear emotional orientation becomes easily accepted by the audience and distributed. Thus, in a situation of a crisis, humans do not become irrational, but they are under the threat of relying on intuition, emotions, and personal bias rather than factual information. AI takes advantage of this human weakness and, modifying the original information, adds emotions and biases to it and thus amplifies it.

Thus, the importance of this case is not only in demonstrating the ways of AI content fabrication but in revealing the intervention of AI technologies in the formation of the meanings associated with events in society’s eyes. Once the original picture was manipulated to create a narrative, the public receives not merely a piece of misinformation but distorted reality. Consequently, what is compromised is not the ability to distinguish information but the ability to build coherent discussions based on facts.

What can be done with AI-generated disinformation

https://www.bing.com/images/search?view=detailV2&ccid=D7SxIy6t&id=336EBF275C58F06FE603A94346EF40A92BBF7BF3&thid=OIP.D7SxIy6tNSs1xQ0GzeKaZwHaHa&mediaurl=https%3a%2f%2fshikakumania.blog%2fgenerative_ai%2fwp-content%2fuploads%2f2025%2f04%2fChatGPT-Image-2025%e5%b9%b44%e6%9c%8830%e6%97%a5-07_37_48.png&exph=1024&expw=1024&q=AI+%e6%b3%95%e5%be%8b%e6%b3%95%e8%a7%84&FORM=IRPRST&ck=E650F3A4FEE12488F9C1464DBAC1E54E&selectedIndex=4&itb=0

In view of all this, the governance of AI disinformation becomes a complex and multi-layered issue, rather than solely a technical one. Therefore, any single solution is not sufficient, and collaboration should take place in order to address the problem (AbuJarour et al., 2024). Firstly, at the individual level, increasing the public’s media literacy and ability to judge content remains the primary task. This implies not only the ability to differentiate between clear falsehoods and the truth but to delay judgment in cases of highly realistic information and perform multi-source verification and critical evaluation.

On the platform level, there is a need to reshape the information distribution mechanisms. Platforms not only distribute content but decide its visibility, and, therefore, there should be a strengthened identification system for AI content that would prevent recommendations and amplify disinformation. Additionally, there is a need to create a more balanced relationship between profit and public interest on the side of the platform to mitigate traffic-driven disinformation.

From the standpoint of the national governance, it becomes necessary to update laws regulating the process of information production by AI technologies. In particular, the currently predominant legal framework, based primarily on humans as creators and transmitters of information, is unable to tackle new technological challenges. Consequently, there is a need to clarify the boundaries of responsibility between AI developers, content distributors, and users of these services and increase the pertinence and enforceability of measures implemented.

In conclusion, it becomes especially important to integrate technological components into the regulation process, including using advanced algorithms and tools to detect, analyze, and track disinformation content through technology-based mechanisms ( AbuJarour et al., 2024).

#DigitalGovernance #DigitalRights #Datafication #Algorithms #SurveillanceCapitalism AI AI Actors AI Content Creation AI Limitations AI Overviews algorithm Algorithmic Goverance Algorithmic Governance algorithms Artistic Value automation ChatGPT content moderation Dark Patterns data scraping Deepfakes Digital Addiction Digital legal responsibility digital policy Digital Privacy digital rights Facebook Google Search hate speech Industry Transformation Instagram Meta online abuse Online harm Online Safety OpenAI personal data Platform platform governance privacy privacy policy Rednote Short Drama social media Social Media Addiction Tiktok Twitter unemployment Youtube

https://th.bing.com/th/id/OIP.qEWq8cSrSnNQeV_eJulMPgHaFd?w=245&h=180&c=7&r=0&o=5&dpr=1.3&pid=1.7

Reference list

AbuJarour, S. A., Qarariah, A., Saadeh, N., & Salem, M. (2024). AI, Misinformation, and Fake News: A Literature Review of Ethical and Technical Approaches. Finance and Law in the Metaverse World: Regulation and Financial Innovation in the Virtual World, 641-652.

Botha, J., & Pieterse, H. (2020, March). Fake news and deepfakes: A dangerous threat for 21st century information security. In ICCWS 2020 15th international conference on cyber warfare and security. Academic Conferences and Publishing Limited (p. 57)

 Crawford, Kate (2021) The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. New Haven, CT: Yale University Press, pp. 1-21.

Milosev, P., & Ackovska, N. (2011, May). AI planning for organizing personal schedules. In 2011 Proceedings of the 34th International Convention MIPRO (pp.  988-992). IEEE.

Pasquale, Frank (2015). ‘The Need to Know’, in The Black Box Society: the secret algorithms that control money and information. Cambridge: Harvard University Press, pp.1-18.

Park, H. J. (2024). The rise of generative artificial intelligence and the threat of fake news and disinformation online: Perspectives from sexual medicine. Investigative and Clinical Urology, 65(3), 199.