When hate content on TikTok can not only gain views and circulation but also sales and profit, online hate speech is no longer just an issue of harmful content; it is digital governance problem that requires serious response.
In October 2025, the UK’s Bureau of Investigation Journalism (TBIJ) reported that some TikTok creators were using AI-generated racist violent videos to attract views and followers. One of the videos showed soldiers opening fire on migrant boats at sea, accompanied by the caption “Fire! Aim at that rubber boat!”, and received nearly 500,000 views.
Some users who ran Tik Tok shops even used the attention gained by these racist videos to sell goods, taking a 20% share of each sale (TBIJ, 2025).
These videos clearly violated TikTok’s policies on violence and hate speech, yet they accumulated nearly 8.5 million views. Some of these videos were only removed after TBIJ contacted TikTok.
This article uses the TikTok AI video case to explore the hidden platform mechanisms and governance dynamics behind it.
The case matters not only because it demonstrates the existence of hate content on the platform, but also because it clearly illustrates that hate speech does not simply appear and get removed. Instead, it may first be amplified by algorithms, then reinforced by engagement logics, and eventually even turned into profit, creating an environment where hate content can more easily survive, circulate and profit.
Therefore, the case should not be regarded as an isolated content moderation failure, but rather as a digital platform governance issue: when a platform can continuously profit from public attention, should it also bear more explicit responsibility for the magnified social harm?
What is hate speech?
Before asking whether Tik Tok’s governance of hate content has loopholes, we first need to be clear about what hate speech actually is. Is hate speech simply a matter of swearing or hurting others’ feelings? In fact, not quite.
As Sinpeng et al. (2021) explain, hate speech is actually a systemic form of discrimination that take place in public spaces. In other words, what is said privately is not within the scope of this discussion. It usually targets specific groups, especially the marginalized ones in social or political life, such as ethnic minorities, LGBTQ+ groups, religious minorities, and so on.
More importantly, the harm caused by hate speech is not a one-off event, but an ongoing trauma that is repeatedly denied and belittled with words, even if no one physically acts on it.
For this reason, many scholars believe that hate speech should be addressed through policies and regulations.
The online hate speech we are going to explore today has several additional features — anonymity, a sense of validation, and easy circulation.
Online anonymity makes hate speech easier to express as speakers do not have to bear responsibilities for the harm they cause. At the same time, they need and enjoy to have their viewpoints being recognised by others. Moreover, due to the fact that social platforms do not have the physical limitations of offline spaces, hate speech often spreads more easily. Lastly, such speech rarely exists alone but is intertwined with misinformation, political extremism, and hostile religious environments.
The platformed hatred: It’s not just because of users
It is tempting to see hate speech as a result of a few prejudiced and malicious users. But hateful content does not spread on social media simply because of them. Platforms themselves also play an important role in producing, amplifying, and sustaining it.
The concept of “platformed racism” proposed by Matamoros-Fernandez (2017) is very useful in this discussion. She uses the term to describe a new type of racism shaped by the cultures, rules, business models, and technical functions of social media platforms. She also analysed the case of Australian AFL star Adam Goodes suffering from racial discrimination across social media platforms and explained how platformed racism works in practice.
On Twitter (now X), the “sensitive media” filter was used to disguise racist memes and avoid being flagged. On Facebook and YouTube, liking or even just watching racist content about Goodes could result in recommending more similar content.
In these cases, sharing and liking helped legitimise racist discourse (Beer, 2016) and feed the platform algorithms that determined what content would be recommended next.
TikTok, on the other hand, presents an even more extreme version of “platformed racism”.
Unlike Twitter and Facebook, TikTok’s is entirely driven by algorithms. From the moment a user opens the app, the For You Page starts to push content. Even brand-new accounts with no prior activities can be recommended racist and Islamophobic content within minutes (Best for Britain, 2025).
This means that TikTok’s algorithm is not simply amplifying existing hate; it is actually proactively pushing hate content to users, even if users have never expressed any interest in it.
As Matamoros-Fernandez (2017) pointed out: Platformed Racism challenges the claim that social media platforms are neutral spaces. TikTok’s recommendation algorithm pushes this non-neutrality even further.
Algorithms + Profits: A Powerful Booster for Toxic Culture
If algorithms explain why hateful content is more likely to be seen, then the profit mechanisms explain why it keeps coming back.
TikTok is not simply a platform for users to express themselves. It is more like an attention-driven space built on watch time, interaction, and conversion rate.
In such a space, the most valuable content is not the most genuine, rational, or publicly valuable. It is the content that grabs attention and stirs controversy.
Anger, fear, and hostility are particularly effective at doing exactly that.
The problem with TikTok is that traffic generated by public emotion is not simply symbolic attention. It can also be quickly converted into economic benefits. As seen in the case of TikTok creators attracting followers with AI-generated racist videos, by which they can sell goods by leveraging their engagement and followers base, while both creators and the platform benefit from every transaction.
In other words, hatred is not only highly sharable but also profitable in such platform environment. Once hateful content can bring about views, follower growth and e-commerce conversions, the platform forms a dangerous incentive mechanism. On this platform, creators do not need to truly believe in the hateful narratives they circulate. It may be enough to produce them simply to attract high attention and make money.
It is just a joke? How humour serves as a shield for hate speech
Another reason why hate speech continues to circulate on content platforms is that it rarely appears in the form of open and explicit attack. Instead, it is often disguised in jokes, satire, and memes.
What makes this especially troubling is that, while platforms have a set of regulations for explicit harmful information, they remain a less clear boundary between humour and harmful expression, which is a major blind spot in the governance of hate speech (Matamoros-Fernandez, 2017).
Guan and Chen (2026) also found that, in China, hateful discourse online often appears through humour, irony, and metaphor. The seemingly “playful expressions”, but in fact, continuously reinforce hostility and exclusion towards particular groups.
The AI-generated racist videos on TikTok show exactly how this works. These videos do not always appear with a direct slogan expressing “I am here to hate a certain group”. More often, they are wrapped in parodies, entertainment, or exaggerated performances, and thus are frequently labelled as funny content by platform. That makes these videos more likely to slip through the grey areas of platform governance than explicit hate slogans, and are more likely to be consumed and shared by ordinary users as “just a joke”.
The problem is that once hatred is turned into entertainment, it is easier to lose the sense of alarm. Hatred without alarming often spreads even faster.
Hence, platforms urgently need to enhance the way their systems identify “disguised hatred” and develop a more detailed governance mechanism on unnoticeable online attack.
How does hate speech affect us
When this type of platform mechanism persists over time, what it brings is no longer just the harm of individual videos but a deeper erosion of social life.
The harm of hate speech has never limited to the individual level. As UNESCO (2023) notes, hate speech not only harms individuals and incites violence, but also attacks inclusion, diversity, and human rights, weakens social cohesion, erodes shared values, stability, and the realization of fundamental rights.
Another case on TikTok is a typical example. After the Southport killings, rumours circulated online describing the suspect as a Muslim migrant, fuelling a large wave of anti-Muslim abuse on social platforms and evolved into offline hostility against Muslim and immigrant communities (Full Fact, 2025).
In other words, when the platform continuously rewards content with high emotions, and high hostility, what they amplify is not just a series of harmful posts, but an entire public culture that normalizes exclusion, humiliation, and dehumanization. The social cost of such an environment is greater than “content violation” can capture.
Lastly, a study by Australia eSafety Commissioner (2025) worth a reference. They found that over 63% of online experiences take place on social media. Among adults who had personally experienced online hate, 54% believe they have at least one significant negative impact, and 34% said it had seriously impacted their mental or emotional health, etc.
eSafety Commissioner
How internet platforms should be governed together
If platforms profit from public attention, they cannot pretend to be neutral channels simply for content flow. Since they decide what to be seen, they also shape how people feel, judge, and treat others. If kind of influence can be converted into profits, then it should always come with social responsibilities.
From a governance perspective, actions should be taken from three levels at the same time.
The government needs to establish a governance framework
One of the biggest governance gaps today is insufficient transparency of the platform algorithmic recommendation mechanism. People do not know which contents will be recommended, amplified, or even converted into profits (Full Fact, 2025). In this regard, it’s not something that can be left to platform’s goodwill alone. Governments need to establish third-party review auditing mechanisms.
The European Union’s Digital Services Act (DSA) (2026) offers one possible model. As the first supranational regulatory framework to enhance the transparency of algorithmic mechanisms, it shifts attention beyond individual pieces of content and towards the wider logic of platform governance.
Under the framework, platforms with over 45 million monthly active users will be regarded as hyperscale online platforms or search engines and must comply with the strictest regulations in the DSA, including reporting illegal content, paying attention to the most basic rights of the public, and maintaining transparency in advertising, recommendation systems, and content moderation.
Meanwhile, the delegated act for Data access adopted by the European Union in 2025 further allows qualified researchers to obtain internal data to study these systemic risks and how those risks might be mitigated (European Commission, 2025).
What matters about the framework is that it shifts the regulatory focus from content governance to “whether the overall system of the platform is continuously amplifying the harm”.
How does the platform balance profit and social responsibilities?
For a highly algorithm-driven platform like TikTok, the problem is never just harmful content itself. The deeper problem is about distribution, expansion, and benefits conversion.
Therefore, if platforms truly want to balance profitability and social responsibility, they need to admit that not all engagements deserve to be rewarded.
When hatred, humiliation and incitement generate more clicks than reason and restraint, the platform can no longer pretend to be just a neutral role. At the very least three changes should be institutionalised.
First, enhance the transparency of the recommendation system, so that external researchers, regulators, and users can clearly understand what type of content are systematically circulated.
Secondly, cut off the direct connection between hateful content and monetization, especially in terms of e-commerce commissions, creator sharing, and advertising revenue.
Finally, regarding the “humour” and “meme” hate expressions that touch upon the governance grey area, platforms should refine the governance policies and separately disclose the handling logic and case types of harmful humorous/meme content in the policy transparency report.
Although the user’s responsibility is the smallest, it does not mean they have no responsibility
We are in a time where we’ve sort of accepted the unrestricted, unregulated mining of the human consciousness, the harvesting of human attention. We are the resource and I think it takes its toll.
In attention economy, attention itself becomes an important resource for online platforms and other industries compete for and monetise. It is exactly why ordinary users are not just bystanders. What we engage with, what we pause for, and what we share all help shape the platform environment around us.
Therefore, as we live in the digital society, we also need to make efforts to create a healthier digital environment.
Australian eSafety commissioner (2025) suggests that if users encounter online hate, they should first report it to the platform and then seek help from eSafety when the platform fails to handle it. At the same time, actively unfollowing, reducing interaction with harmful content, and blocking accounts that make people feel unsafe. Furthermore, it is also suggested that not to repost inflammatory content, supporting reliable news sources, and participating in counter speech under safe conditions.
These actions may seem minor, however, in such attention-oriented era, they are the few initiatives that still hold in our hands.
Therefore, what here truly aims to emphasize is that a platform environment that prioritizes long-term profitability ultimately depletes not only the experience of individual users, but also the trust, cohesion, and participation of the entire society.
When hatred can stably generate traffic and traffic can be smoothly converted into profits, the platform can no longer treat online hate merely as a “problem of managing illegal content”. It has already become a governance issue related to basic rights, public security and the quality of democratic public Spaces. What really needs to be asked is not just whether TikTok has deleted a few videos, but rather: in this environment where attention is currency, what kind of public discourse is the platform rewarding?
References
Beer, D. (2016). Metric power. Palgrave Macmillan. https://doi.org/10.1057/978-1-137-55649-3
Centre for Humane Technology. (2021). The attention economy. https://www.humanetech.com/youth/the-attention-economy
eSafety Commissioner. (2025). Hate in the digital age: Adults’ encounters with online hate. Australian Government. https://www.esafety.gov.au/sites/default/files/2025-04/Hate-in-the-digital-age-adults-encounters-with-online-hate.pdf
European Commission. (2025, July 2). Commission adopts delegated act on data access under the Digital Services Act. Shaping Europe’s Digital Future. https://digital-strategy.ec.europa.eu/en/news/commission-adopts-delegated-act-data-access-under-digital-services-act
European Commission. (n.d.). DSA: Very large online platforms and search engines. Shaping Europe’s Digital Future. https://digital-strategy.ec.europa.eu/en/policies/dsa-vlops
Guan, T., & Chen, X. (2025). Threat perception, otherness and hate speech in China’s cyberspace. Journal of Contemporary China, 35(158), 1337–1352. https://doi.org/10.1080/10670564.2025.2475051
Manzi, Z., & Rose, H. (2025, July 29). What the UK riots taught us about social media failure. Full Fact. https://fullfact.org/crime/what-southport-taught-us-about-social-media-failure/
Massanari, A. (2017). #Gamergate and the Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3), 329–346. https://doi.org/10.1177/1461444815608807
Matamoros-Fernández, A. (2017). Platformed racism: The mediation and circulation of an Australian race-based controversy on Twitter, Facebook and YouTube. Information, Communication & Society, 20(6), 930–946. https://doi.org/10.1080/1369118X.2017.1293130
McIntyre, N. (2025, October 16). New AI video tools are fuelling violent racism on TikTok. The Bureau of Investigative Journalism. https://www.thebureauinvestigates.com/stories/2025-10-16/new-ai-video-tools-are-fuelling-racism-on-tiktok
Rodrigues, L. (2025, December 5). We went undercover on far-right TikTok. Here’s what we found. Best for Britain. https://www.bestforbritain.org/far_right_and_tiktok
Sinpeng, A., Martin, F., Gelber, K., & Shields, K. (2021). Facebook: Regulating hate speech in the Asia Pacific. Department of Media and Communications, The University of Sydney, and School of Political Science and International Studies, The University of Queensland. https://doi.org/10.25910/j09v-sq57
Spring, M. (2026, March 16). Meta and TikTok let harmful content rise after evidence outrage drove engagement – whistleblowers. BBC News. https://www.bbc.com/news/articles/cqj9kgxqjwjo
UNESCO. (n.d.). What you need to know about hate speech. https://www.unesco.org/en/countering-hate-speech/need-know
Be the first to comment