Image from ABC News (2025)
Enter TikTok or X, just swipe for several minutes, and you will encounter harmful content such as disinformation, cyberattacks, and blatant expressions of hate speech. The emergence of such phenomenon is no longer accidental or the extreme behavior of individuals. However, it will eventually become an everyday life situation within the digital domain.
Although in recent years, the major platform has improved its supervisory policy through introducing better technologies of algorithm audits and making promises to ensure a safe cyberspace, such harmful phenomena still exist extensively. Such contradictory situation inevitably prompts one critical question to rise: why is it impossible to eradicate harmful content despite increasingly strict regulation by the platform in recent years?
The present article argues that the difficulty of eradicating harmful content is not solely because of the platform’s “weak management.” Rather, this issue is an inevitable consequence of a structural contradiction – a mismatch and conflict between platform governance system and algorithm-digital culture. Based on this core idea and current case study of X platform, the present paper seeks to explain the persistence of harmful content from the angles of definition, implementation, algorithms, and digital culture.
The Problem of Definition: What Counts as “Harmful”?
To tackle the issue of problematic content on the Internet, it is necessary to first define another complicated question: what is “harmful”?
Hate speech can be generally defined as an act of attack, degrading or dehumanizing a particular group based on identity categories like race, gender, religion or sexual orientation. However, according to Terry Flew, digital platforms need to translate these complex and contextualized social notions into operational policies. In doing so, some important contexts and nuances are inevitably omitted.
Moreover, Sinpeng et al. (2021) further emphasized the difference between idealistic and realistic definitions of hate speech. For efficient mass content regulation, the platform will implement more simplified and narrow definitions. Most of the time, companies like Meta or X would apply clearly defined and quantitative guidelines instead of applying contextually complicated judgment criteria.
It resulted in one clear result, namely, the emergence of what scholars called the “governance gap.” Not all content that is harmful from a sociocultural perspective would necessarily be in violation of platform guidelines. In this regard, languages, sarcasm, plot, or symbolism with implied discrimination could promote harmful ideology without breaking any guidelines.
In other words, the cause of the existence of such dangerous content is not that the platform neglects them, but simply that they exist beyond the scope of platform definition.
The Enforcement Problem: Why Rules Are Not Enough
Even if the platform can define harmful content more clearly, how to effectively implement these rules globally is still a huge challenge.
At present, most platforms rely on the hybrid mode of “manual audit + automation system”. The latter feature of artificial intelligence auditing includes high efficiency, but it is hard for it to judge correctly in case of complicated situations. Although manual auditing is more advantageous in understanding the context, its processing capacity is obviously limited in the face of a large amount of content generated every day. At the same time, auditors also need to bear the psychological pressure brought by long-term exposure to extreme content.
As Flew (2021) pointed out, the platform has in fact assumed a role like that of a “regulatory agency”, but it lacks the transparency and accountability mechanism that should be in the traditional governance system. This situation creates an environment where the execution of the policies appears to be highly unstable.
For instance, there can be notable variations in the review criteria depending on the geographic locations and languages used. Although some kinds of content may be deleted under certain conditions but stay untouched under others, this type of variation does not only lower the effectiveness of the mechanism itself but also creates doubts about its fairness.
Like any other instance of X, the matter becomes more urgent in this case. Due to the decrease in the number of members of the audit group, it is challenging for the website to handle the negative content, resulting in an increase in illegal or questionable materials.
Thus, it may be said that the establishment of such rules alone cannot solve the problem; in the absence of an effective method of implementation, the perfect policy remains a mere policy.
The Algorithmic Layer: When Platforms Reward Harm
If the policy decides “what can exist” and the execution decides “what will be deleted”, then the algorithm decides “what will be seen”.
Source from Straight Arrow News – SAN (2026)
At this level, the problem becomes more complex and crucial.
From what is highlighted in the concept of “toxic technology culture” mentioned by Massanari, Adrienne (2017), the platform is not a neutral medium of communication but uses algorithms and other elements to control its users’ behaviors.
It is the underlying assumption in almost all social networking sites that the best way to make use of their full potential would be through increased user engagement. Hence, the system would favor people that can trigger feelings of anger and outrage.
This creates an “amplification effect”: the more outrageous the material, the easier it is for the material to gain publicity and, thus, increase its interactions. The problem is that in this situation, not only is the objectionable material not discouraged, but sometimes the material gets rewarded in some form or other.
Thus, the occurrence of problematic content is not just due to lack of regulation but also because of inevitability within the platform structure itself.
Why Hate Speech Is Rising on X
Have you been using X lately? If yes, then you would definitely realize that something new has happened. The general situation has become harsher; improving your dialogues has become much easier, and the content that you hate seeing is very common. But one thing that cannot be denied is that this situation is not just personal for you, since the increase in hate speech on X was more than 50 percent. (Manke, 2025)
Source from Berkeley News (2025)
This shift could mostly be attributed to the time after Elon Musk became the new owner of X. Since then, the concept of “freedom of speech” began to be stressed, while the strict monitoring system was being relaxed.
On the surface, this is to make the platform more open. But the actual effect is not the same. Although X claims that users are exposed to less harmful content, researchers point out that this statement lacks transparent data support. (Morris-Grant, 2025)
The gap between the “platform statement” and the “user’s actual feeling” is the key to the problem.
First, when the rules on harmful content become looser or vaguer, the boundaries of speech will change rapidly (Sinpeng et al., 2021). The content that could have been deleted began to be preserved and continued to spread, and even gradually “normalized”.
Secondly, audit is not only a question of whether there are rules, but also whether there is an ability to implement them. When the audit resources are reduced, a large number of harmful contents will be directly “leaked” and stay longer.
Finally, it is the role of the algorithm. Even if you don’t take the initiative to read the controversial content, the platform may push it to you. Content that can stimulate strong emotions (such as anger or conflict) is often easier to interact with and be recommended (Massanari, Adrienne, 2017). In other words, hate speech is not only not suppressed, but is “pushed to the front” to some extent.
Looking at these factors together, it is not difficult to understand the rise of harmful content on X. This is not caused by a wrong decision, but the result of the superposition of multiple changes: the rules have changed, the execution has become weaker, and the algorithm is still rewarding the content that “can trigger interaction”.
Because of this, this is not only X’s problem, but a bigger signal: when different parts of platform governance do not work together, the problem will quickly amplify.
Where is the breakdown?
On the whole, the persistence of harmful content is not caused by a single cause, but the result of the joint effect of multiple levels.
At the policy level, the platform’s definition of harmful content is often too narrow to cover complex and contextual harm. As Sinpeng et al. (2021) pointed out, there is a clear gap between the ideal definition of hate speech and the actual implementation of the platform, which makes it possible for much “harmful but not illegal” content to exist.
At the execution level, the audit mechanism is often unstable and resources are limited. Even if the rules exist, the platform may not be capable enough to continue to execute in a large-scale content environment, resulting in a large amount of harmful content being visible for a long time.
At the algorithm level, the recommendation system tends to amplify emotional and controversial content. As Massanari, Adrienne (2017) pointed out, the platform structure may give rise to a “toxic technology culture”, so that anti-aggressive content is not only disseminated, but also rewarded.
At the cultural level, the acceptance and even promotion of aggressive conduct within certain online forums have also contributed to the propagation of harmful content.
More significantly, these dimensions are not unrelated to each other, they complement each other: narrow definitions complicate implementation, lack of proper implementation leads to survival of undesirable communities, and all this is then reinforced by the algorithm.
What is missing?
Going further, the issue is not just that the system is “a little chaotic”; rather, different aspects of the system have different agendas, and those agendas are not always in harmony.
For example, platforms can state that they aim to limit problematic material, but in doing so, they also need to maintain user engagement and interactivity. Both goals may seem to run contrary to each other; content that keeps the user engaged may not be safe for him or her.
The more insidious issue is that even if the system acknowledges that the content is potentially dangerous, there is an inconsistency with its handling. For instance, some content is immediately removed, others are allowed to persist, and some are in limbo, and the platform itself does not know what should be done about them.
In other words, the issue here is not only about the lack of technology; it also stems from how the structure of the system operates. This encompasses questions like “What is harm?” and “What takes precedence?”, among others.
This is the reason why merely fixing a portion of the system won’t fix anything either. As long as there is no adjustment to the incentive system, harmful content will still come up in other ways.
What kind of governance is needed?
Since the problem is a “misalignment of systems”, the solution lies not only in creating new rules, but in optimizing the operation of the entire system.
At the level of the platform itself, transparency plays an extremely important role here. Users need to understand exactly what and why the content was deleted or recommended. Instead of making everything mysterious and even arbitrary, it would be nice to understand exactly what the platform wants to optimize. Since the main objective remains “interaction”, it means that emotional and provocative content will still rule.
As for regulation, there should not be control over content directly, but responsibility. Responsibility for what? For the results of the functioning of the system itself.
Lastly, governance should not be done exclusively by the platform and the state but also involve researchers and users themselves.
In general, the essence of the matter is not “full control” but the creation of a consistent system without contradictions.
Source from PCMag Australia (2023)
Overall, hate speech and network harms within X are not the consequences of one failed policy, but rather the results of structural issues within the platform’s governance model. If the audit loses its power, and the algorithm still gives preference to the dissemination of interactive posts, then there is a risk of the proliferation of harmful content. The case of X demonstrates that addressing network harms should not depend on content deletion alone but requires a reconsideration of speech definition within the platform.
Reference
Flew, Terry (2021) Hate Speech and Online Abuse. In Regulating Platforms. Cambridge: Polity, pp. 91-96 (pp. 115-118 in some digital versions)
Manke, K. (2025, February 13). Study finds persistent spike in hate speech on X – Berkeley News. Berkeley News. https://news.berkeley.edu/2025/02/13/study-finds-persistent-spike-in-hate-speech-on-x/
Massanari, Adrienne (2017) #Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3): 329–346.
Morris-Grant, B. (2025, February 12). Elon Musk promised to stamp out hate speech and bots on X. New research shows neither happened. Abc.net.au; ABC News. https://www.abc.net.au/news/2025-02-13/hate-speech-bots-twitter-report/104923196
Sinpeng, A., Martin, F., Gelber, K., & Shields, K. (2021, July 5). Facebook: Regulating hate speech in the Asia Pacific. Final Report to Facebook under the auspices of its Content Policy Research on Social Media Platforms Award. Dept of Media and Communication, University of Sydney and School of Political Science and International Studies, University of Queensland. https://r2pasiapacific.org/files/7099/2021_Facebook_hate_speech_Asia_report.pdf
Be the first to comment