The Hate Algorithm: How Platform Capitalism Turns Toxicity into a High-Profit Product

Digital Policy & Governance
By Qile Pan
April 4, 2026
华尔街日报 Facebook Files 报道的新闻标题截图

 

Fig 1. The Wall Street Journal’s 2021 investigation revealed that algorithm changes designed to boost engagement actively rewarded anger and toxicity.
Source: The Wall Street Journal

Ok, if you’re anything like me, you’ve probably been spending waaaay too much time scrolling social media lately, feeling exhausted and depleted and furious when you finally shut off the app. Have you noticed that the posts that tend to go super viral- the ones with hundreds of thousands of shares and thousands and thousands of comments- are always the ones that are really rude, really mean, or always the ones that are the most provocative, you know?

Any time a huge online witch hunt occurs, or someone says something really offensive about a marginalized community, the first instinct is to sigh, “Humans suck,” and think that humans are just generally the worst. We figure that internet anonymity has just brought out the worst in society. People seem to think of internet trolls as a virtual social commentary on the ugliness of the human condition and the way that the world is divided into people who think alike and people who think differently.

But what if I told you that we are looking at this entirely the wrong way?

Through my recent deep dives in my Digital Policy & Governance class, I’ve realized something that fundamentally changed how I view every single post on my newsfeed.

My thesis for today is simple, but honestly, it’s a little terrifying: Online hate speech isn’t just a reflection of human ugliness in reality; under today’s “platform capitalism,” it is a highly profitable product deliberately amplified by algorithms to harvest our attention and click-through rates.

Get yourself a coffee and sit back, because today we are taking a dive on Big Tech. We’re going to delve into some fascinating academic research with a particular focus on the Asia-Pacific, and we’re going to break down a major real-world drama from the beginning of 2025 that demonstrates just how vast and lucrative this algorithmic rabbit hole goes.

The Attention Economy: Why Anger Pays the Bills

To truly understand why hate speech grows like a noxious weed in our digital gardens, we have to stop looking at the users and start looking at how social media companies actually make their money. We often forget a crucial rule of the internet: if you aren’t paying for the product, you are the product. We aren’t the customers of platforms like Facebook, Instagram, X (formerly Twitter), or TikTok. Advertisers are the customers.

And thus we reach the core concept of the state of attention economy. Within the framework of platform capitalism, the big technology companies do not sell information or intel; they sell behavior influencing. In order for the companies to substantially increase their profits, they must try their hardest to grab your attention, and keep it confided on the screen for as long as is possible. Every additional second of scrolling means one more advertisement displayed, and one extra half cent in revenue.

So, how do you keep scrolling for the most extended period? Definitely not by showing soothing, indifferent, or cheerful material. Tech companies use “engagement-based ranking” algorithms. These algorithms are extremely complex and programmed, especially for one cold-blooded purpose: keep serving up the content that gets the most traffic, likes, shares, comments, and watch time at the top of your feed.

Here’s the psychological trick that algorithms are taking advantage of: from an evolutionary perspective, something that threatens danger or makes us feel bad gets a strong reaction from humans. You can scroll past a photo of your friend’s healthy lunch, but if you see something that’s infuriating, sad, scary, or extremely offensive, you’ll definitely stop. You’ll stop, you’ll read, you’ll get mad – and, most importantly, you’ll comment back angrily.

Ethical considerations are of no importance to the algorithms, yet they surely know about numbers. They understand that outrage equals the attention they hope to get, and each attention is a business opportunity for them. The algorithm has no problem with a post that is full of hatred and division; that is the easiest way to become successful for it.

The Asia-Pacific Reality Check: When Algorithms Exploit Local Cultures

Although there is a theoretical impression of a similar problem, which generates solely in American politics, we can bring it much closer to home. The APAC region is where you can observe these engagement-driven algorithms, and this fact makes it clear how they use cultural tensions in a profit-driven manner. Let us examine three specific studies from recent academic literature where this phenomenon was rigorously and comprehensively documented.

To start with, we will look at the online environment in China. At least equally exciting, in a 2026 article in the Journal of Contemporary China, Tianru Guan and Xiaotong Chen declare that we live in the age of identity fundamentalism, where identity politics and culture wars shape the self-justifying us versus them (Guan & Chen, 2026). And in some of their best papers they show us that a sub-part of the attention economy makes the identity crisis even worse (Guan & Chen, 2026). The algorithms used by Chinese digital platforms quickly get it that fear narratives of personal threats causes an explosion of traffic. To take just one example, the algorithms primarily amplify negative stereotypes of country folk based on the household registration system and class or religious anti-foreign attitudes(Guan & Chen, 2026). Why? Because these reinforce oppositional and even hateful narratives. These platforms essentially sell polarization, making the issue worse by remarginalizing the already marginal, making them the most powerful engagement(Guan & Chen, 2026).

Moving down to Australia, the Indigenous peoples are currently suffering a great tragedy due to what scholars Bronwyn Carlson and Ryan Frazer refer to as “platformed racism.” (Carlson & Frazer, 2018). As evidenced in their all-inclusive report published in 2018, page 12, Carlson and Frazer are clearly observing the Social Media scene that is fraught with “racial violence” against Aboriginal and Torres Strait Islander peoples (Carlson & Frazer, 2018). Illustrating the worst part of this tragedy on page 9 of their report, the scholars describe how “Sorry Business,” a sacramental grieving procedure unique to the Indigenous peoples, is constantly disrupted and mocked by trolls on social media (Carlson & Frazer, 2018). In addition, the Indigenous public is often racially verbalized by the trolls who many times claims that they are “too white” to be real Aboriginals (Carlson & Frazer, 2018). Since these acts of racist trolls typically create chatter, angry comments, and clicks, the algorithm helps in propagating them rapidly across these platforms. The Indigenous public is now forced to censor themselves and hide their backgrounds in order to escape the trolls as these Social Media Platforms silently enjoy the fruits of their terrible seeds.

麦考瑞大学关于原住民网络伤害的研究报告截图

 

Fig 2. Research from Macquarie University reveals the disproportionate online harms faced by Indigenous communities.
Source: APO (Analysis & Policy Observatory)

So now, broaden your perspective to the aggregate of the APAC LGBTQ+ community—surely you’re thinking, “Facebook has an intuitively complex AI system to proactively and automatically remove the type of hate speech mentioned, right?” Unfortunately, the situation is vastly imperfect. For example, in their 2021 work, Sinpeng et al. (University of Sydney) reported a “large language gap” in automatic moderation—as highlighted on page 3, while English-language hate speech might be detected relatively well, local slurs and cultural context fall by the wayside in the 2300+ languages all across the Asia-Pacific (Sinpeng et al., 2021).

In an essence, LGBTQ+ group leaders in countries such as Indonesia, the Philippines, and Myanmar face complete exposure to deprivation hate speech, which is an attack intended to take away their rights to live and be who they are (Sinpeng et al., 2021). What makes the situation even worse, the report emphasizes on page 1, when they attempt to make manual reports about hate speech, Facebook’s being totally non-responsive leads to serious reporting fatigue. The users actually drop out of this struggle. Unlike the platform’s profit-centered, automated moderation system, which is blindly insensitive to their distress, and therefore, cannot be comparable in quality of service to the expensive in-context human moderation system, these users are most likely to face danger of exposure to inhumanities being automated moderated.

悉尼大学联合发布的 Facebook 仇恨言论官方报告封面截图

 

Fig 3. Report by The University of Sydney highlighting the disparity in Facebook’s hate speech regulation across the Asia Pacific.
Source: UQ eSpace

The 2025 Meta Meltdown: A Case Study in Profit Over People

大赦国际官网关于 Meta 算法放大罗兴亚暴力的官方新闻截图

 

Fig 4. Amnesty International’s ongoing legal actions against Meta expose the deadly real-world consequences of algorithmic amplification.
Source: Amnesty International

If you need any further proof that Big Tech intentionally opts for more clicks and profit out of their algorithms as opposed to remedying the harm we encounter online, we just have to remember the major, historic scandal that emerged and came to light in early 2025, January to February.

For years, the platforms played pretend on hate speech, doing all that they could (and also not) in order to stop hate speech while gaming the algorithm with the twist of a knob for maximum engagement. In early 2025, however, they were unmasked.

In January 2025, Meta (the corporation behind Facebook and Instagram) declared a substantial change to its content moderation practices disguised as “free expression,” or a return to policy for speech that had previously been moderated (Amnesty International, 2025). Regardless of the name they gave it, the shift was obvious to most human rights organizations: it was to reduce the friction on engagement with the platform – and in the end, to make more money for advertisements.

The response was legal, as well as verbal. It was February, 2025. With the help of Amnesty International, he filed an official “whistle blower complaint” to Security Exchange Commission (SEC) of US about Meta (Amnesty International, 2025).

This 2025 SEC complaint is the smoking gun for our entire thesis. The complaint made it obvious that Meta “knew, or was reckless in not knowing, that its algorithms were calibrated to bolster hate speech that increased user engagement on its platforms.”

The complaint points out that the engagement system of the company was promoting increasing and violent attacks against the Rohingya ethnic group in Myanmar in 2017 since the anti-Rohingyan hate speeches had increased user engagement on the platforms (Amnesty International, 2025).

What is more grave, according to the whistleblower report, Meta is purposely misleading its investors as well as the public regarding the mentioned systemic risks. In 2025, early on, Meta intended to relax its content moderation policy to improve the figures of user engagement. Then again, the activists insisted that this approach would be nine times more likely to lead Meta to repeat the setting of widespread violence.

This new case study validates that hate speech on social media is not a bug. It is not a mistake. It is a normal, extremely beneficial side effect of the platform capitalism business model. Algorithms are literally designed to promote the most emotionally harmful and offensive content because that is what gets people to keep clicking. If a corporation is consciously dismantling safety protocols in light of human suffering that has happened in the past, it’s evidence that the outrage cycle is working as designed.

The Takeaway: Stop Blaming Humanity, Start Fighting the Code

Where exactly does this put us? What would be the main lesson or conclusion to take away from all this academic research and the Meta whistleblower 2025 scandal for us, the ordinary internet users?

The next time you are scrolling through your feed and you see a toxic, hateful comment thread blowing up, I want you to fundamentally change how you react. Stop falling into the trap of thinking, “Wow, people are awful.” Instead, I want you to look at your screen and think, “Well, this algorithm is making a tech billionaire a lot of money off of this outrage.”

It is time to stop playing the futile game of digital whack-a-mole, reporting individual internet trolls to the platform, while the underlying architecture continues to generate trolls. We have been trained to accept human nature as the problem, but human nature is simply the clay that is being used by a multi-billion dollar attention extraction industry.

However, now as digital citizens, we should demand more than just “delete more bad posts.” We need algorithmic transparency. We need government regulations with financial penalties for tech giants, as recommendation engines are in fact designed to recommend hate speech in return for a company’s quarterly advertisement funds, and therefore companies must be financially and legally punished.

If we don’t learn to recognize hate speech for the weapon that it is, purposely engineered by corporations to generate revenue in the attention economy, the internet is always going to be a hateful, toxic environment. Instead of being mad at each other, we should be mad at the algorithm.


References

Amnesty International. (2025). Submission to the Securities and Exchange Commission on Meta’s violation of security laws concerning Facebook’s role in the atrocities against the Rohingya in 2017.

Carlson, B., & Frazer, R. (2018). Social media mob: Being Indigenous online. Department of Indigenous Studies, Macquarie University.

Guan, T., & Chen, X. (2026). Threat perception, otherness and hate speech in China’s cyberspace. Journal of Contemporary China, 35(158), 1337–1352.

Sinpeng, A., Martin, F., Gelber, K., & Shields, K. (2021). Facebook: Regulating hate speech in the Asia Pacific. Department of Media and Communications, The University of Sydney.

Be the first to comment

Leave a Reply

Your email address will not be published.


*