When people talk about online hate, the usual explanation is simple: bad actors post harmful content, platforms fail to remove it quickly enough, and vulnerable groups suffer. This account is not wrong, but it is incomplete. Online harms are not merely a matter of harmful speech or malicious users – it is also a consequence of platform design. Algorithmic selection, engagement-driven visibility, and inconsistent content moderation can amplify hate, reward anger, and place the burden of regulation on vulnerable groups (Just & Latzer, 2016). Online hate is not merely posted; it is organized, amplified, and unevenly governed by the platforms themselves.

This matters because the internet does not provide a neutral space where all speech circulates equally. Instead, what people see is increasingly shaped by algorithmic selection: automated systems determine what is relevant, worthy of attention, and how content is positioned in feeds, search results, trending lists, and recommendations. As Just and Latzer (2016) argue, this process of algorithmic selection has become a central force shaping social order and shared reality. These systems affect not only the information people encounter, but also shape their understanding of the world and their actions within it. Compared with traditional media, this algorithmic construction of reality creates more personalization, commercialization, and inequality, while making the system less transparent, controllable, and predictable.

From this perspective, online hate is not merely something that platforms fail to stop; it is something that platform systems can actively organize, amplify, and normalize. If the design of a platform rewards attention, outrage, and constant engagement, then harmful content spreads not in spite of the platform’s logic, but because of it. Therefore, online harms should be understood not only as resulting from individual bad behavior, but also as a consequence of platform architectures that determine visibility and unevenly distribute risk.

Online harms are not only speech problems. They are visibility problems.

On many social media platforms, highly controversial content often triggers a large number of replies, shares, and prolonged engagement. The more interactions a post receives, the more likely it is to be further promoted by algorithms to a wider audience. This creates a dangerous feedback loop: anger drives participation, participation enhances visibility, and visibility in turn fuels more anger. Consequently, in systems where dwell time, engagement, and return visit rates are core metrics, hate speech may not only become profitable but even serve as the “natural fuel” for this logic.

Reddit offers a powerful example. Its design, algorithm, governance, and culture collectively nurtured antifeminist and misogynistic activism. Reddit fosters toxic technocultures not simply because extreme users exist on the platform, but because its design, policies, and community culture reinforce each other: karma points, default sorting, and the aggregation mechanism of /r/all amplify highly engaging, controversial, and hostile content, making it appear more mainstream and representative. Exposure of spaces that are inclusive of minorities or females on the platform often invites harassment and mobbing. At the same time, under the guise of being a “neutral platform,” Reddit is reluctant to intervene deeply, while its reporting and moderation tools remain limited. Coupled with the ease of recreating accounts and subreddits, and the heavy outsourcing of moderation responsibilities to volunteer moderators, this makes it easier for harmful communities, such as those that are misogynistic or racist, to persist and spread (Massanari, 2017). A platform might insist it simply hosts user speech; however, its ranking systems, engagement incentives, moderation rules, and default settings all influence which voices get heard. As Massanari (2017) argues, platform politics are built into design: algorithms, scripts, policies, and norms shape user behavior – and are shaped by it in return.

Platform design does not create hate from nothing, but it can make hate easier to scale.

What keeps these toxic dynamics alive is not just visibility, but social reward. Online hostility can create a sense of solidarity among participants, especially when attacking a target becomes a way of proving shared values and group loyalty. On the one hand, social media platforms make it easier for people to find like-minded individuals. This, in turn, reinforces their sense of identity and belonging. On the other hand, these same platforms foster exclusion, hostility, and aggressive interactions. For example, Morales et al. (2025) found that while digital platforms promote community and identity, they also encourage harmful exclusionary behaviors. Furthermore, hate speech operates both as a mechanism for harming others and as an indicator of othering and belonging. Thus, digital platforms do more than connect people; they also determine who counts as “us” and who is labeled as “them.” Often, people come to social media seeking connection, recognition, and affirmation from like-minded individuals. However, these connections form not only through shared support or interests, but sometimes through shared hostility. In this way, harmful remarks help draw moral boundaries, allowing group members to show they share the same norms, values, and enemies.

In other words, hate speech often performs two social functions simultaneously: internally, strengthening a sense of belonging; externally, clearly identifying the target of exclusion. Within a group, harmful remarks help members draw clear moral boundaries. Online collective harassment rarely occurs without cause. It typically begins when a member accuses a target of “crossing the line” or “violating group norms”, followed by others expressing moral anger through insults, mobbing, and humiliation. This process is not only punishing the “violators” but also proving to one another that we follow the same rules and belong to the same group. Meanwhile, harmful remarks are also often used to identify and attack external members. Social media provides a space for continuous interaction within groups, making it easier for hostility to accumulate and spread, and allowing them to portray external groups as threatening, inferior, or even unworthy of sympathy. Once such a dehumanizing narrative takes shape, attacks and harms can be more easily rationalized (Morales et al., 2025).

Hate speech online is not just an out-of-control emotional outburst. It also serves as a “mode of connection” in today’s digital environment – allowing people to form and reinforce group identity through shared anger, exclusion, and collective attacks. More troubling, platforms rarely curb these antisocial behaviors meaningfully because outrage, conflict, and emotional polarization most strongly drive engagement, attention, and time spent online.

Figure 1 Chen Yuxi competing on May 3, 2025.

Note. Photograph by Luo Yuan. From “Diving—World Cup Super Final: Chen Yuxi and Quan Hongchan claim gold and silver in women’s 10m platform,” by Xinhua News Agency, 2025, May 4, Xinhua Net. https://www.news.cn/sports/20250504/f987e5a8555b4ec6ace3065b08e23ead/c.html. Copyright 2025 by Xinhua News Agency.

Chen Yuxi and Quan Hongchan’s fandom conflict offers a clear example of this dynamic. As the two dominant athletes in women’s 10-meter platform diving (Xinhua News Agency, 2025), their intense visibility makes them ideal figures for rival fandoms to turn into symbols of “us” and “them.” In practice, some fans reinforce group belonging by attacking the rival athlete and her supporters, framing them as illegitimate, suspicious, or morally inferior (Yulezibenlun, 2024). For example, when Chen Yuxi wins, some extreme fans may frame Quan Hongchan’s absence due to injury as “avoiding competition”. Conversely, when Quan receives more attention, some of her supporters may dismiss Chen’s success as the result of “connections” or “behind-the-scenes manipulation.” Rather than being random outbursts, these attacks are often coordinated practices – involving the collection of “black material,” the production of insulting content, and cross-platform circulation – which can escalate into collective harassment (Ministry of Public Security of the People’s Republic of China, 2025). Even more consequentially, “counter-cyberbullying” typically deepens the spiral of conflict: once one side sees itself as victimized, it retaliates in the name of “justice,” turning abuse into a cycle of reciprocal revenge (Su, 2026). Ultimately, platforms struggle to contain this not only because of the volume of content but also because outrage, conflict, and polarization are exactly the kinds of interactions that drive engagement.

More crucially, hate is contagious and escalates, leading to a typical form of “retaliatory cyberbullying” – if you insult my idol, I’ll insult you back twice as hard; today I’m the victim, but tomorrow I’ll become the bully, waving the banner of “fighting back.” With each round of retaliation, hatred snowballs, growing ever larger, as both sides remain convinced that they are on the side of justice and that the other is the villain. This also explains why platforms are consistently unable to curb this chaos. The essence of algorithms is to keep users on the screen, and what grabs attention most? Anger, conflict, and polarization – these are exactly the core fuels of the platforms’ engagement economy. Research has found that content that provokes hatred is more likely to trigger interaction. Algorithms will identify it as a “high-value emotion” and then recommend it to more people (Brady et al., 2021). Every comment, share, and like sparked by a flame war is the data the platform values most.

This is not just a moderation problem. It is a governance problem.

Once we recognize that design can amplify hate, the limits of moderation alone become clear. A platform can delete abusive posts but may still use ranking, recommendation, and interaction systems that make harmful content easy to find and hard to avoid. For these reasons, online harms are issues of digital policy and governance, not just debates about speech and censorship. 

The deeper question is: who gets to organize public attention online? Platforms claim they are neutral. However, they govern visibility through proprietary, commercially driven systems. Algorithmic governance by private, opaque companies reduces transparency and accountability. This matters because decisions about ranking and relevance shape public discourse, norms, and the spread of harm (Just & Latzer, 2016).

Crawford’s argument is key: AI systems are embedded in political and economic structures, so online harms are not just side effects but results of optimization, incentives, and power. Building on this, Crawford asks: optimized for whom, by whom, and who decides? When platforms are rewarded for engagement and growth, harmful but compelling content fits their logic. Therefore, safety measures are merely reactive patches on systems never intended for public well-being (Crawford, 2021). Seen this way, online hate results from the political economy of visibility: some groups are more publicly exposed, some harms are less visible and harder to detect, and some communities shoulder more unpaid moderation. The question is not just whether platforms moderate enough, but whether they should govern public communication through systems that lack democratic scrutiny.

Current policy debates are starting to move in this direction.

Recent policy developments suggest that regulators are also beginning to recognize this shift from content to systems. Australia is shifting to treat online harm as a systems issue instead of just targeting individual bad content. eSafety’s Basic Online Safety Expectations require providers to assess risks, implement safety measures, and monitor platform safety to keep users safe. These obligations are supported by a Compliance and Enforcement Policy and a broader transparency framework under the Online Safety Act. The focus is no longer solely on removing harmful content after the fact, but on whether platforms have risk-reducing systems and governance from the start (Australian Government | eSafety Commissioner, 2022).

China has adopted a more systemic style of governance. It frames issues less as “hate speech” and more as illegal and harmful information, cyber violence, algorithmic recommendations, deep synthesis, and generative AI. Provisions on the Management of Algorithmic Recommendations in Internet Information Services require providers not to use recommendation systems to share legally prohibited information. They also require platforms to establish internal mechanisms for algorithmic review, information review, security assessment, monitoring, and the regular verification of algorithms, models, data, and outcomes (Cyberspace Administration of China, 2022). This shifts regulation beyond simple takedown logic, focusing on governing recommender systems themselves.

Building on this systemic approach, China has introduced governance rules targeting cyber violence. In its official interpretation of Provisions on the Governance of Cyber Violence Information, the government calls cyber violence a social scourge and says prevention must come first. The provisions require platforms to set up early-warning models that incorporate factors such as incident type, target, number of participants, information content, posting frequency, dissemination stage and context, and user reports. These models help flag risks quickly, indicating that the key to governing cyber violence lies in addressing its source (Cyberspace Administration of China, 2024). Like Australia, Chinese regulators target both harmful content and the systems that recommend, generate, organize, and circulate it. 

Conclusion

Platforms must stop treating hate as a mere content issue and acknowledge it as a critical design problem. Otherwise, efforts to ensure online safety will always be inadequate and reactive. Online hate is not the product of isolated bad actors; it is systematically amplified by platforms’ design choices. Research into algorithmic governance, toxic technocultures, and hate speech regulation confirms a stark reality: ranking, recommendation, aggregation, and automated moderation fundamentally shape online visibility and determine who suffers most from hate (Just & Latzer, 2016). If platforms and regulators persist in viewing hate primarily as a content issue while ignoring core design and governance flaws, their response to online harms will remain fundamentally limited.

References

Australian Government | eSafety Commissioner. (2022). Regulatory guidance. ESafety Commissioner. https://www.esafety.gov.au/industry/regulatory-guidance?utm_source

Bolsover, G., & Howard, P. (2018). Chinese computational propaganda: automation, algorithms and the manipulation of information about Chinese politics on Twitter and Weibo. Information, Communication & Society, 22(14), 2063–2080. https://doi.org/10.1080/1369118x.2018.1476576

Brady, W. J., McLoughlin, K., Doan, T. N., & Crockett, M. J. (2021). How social learning amplifies moral outrage expression in online social networks. Science Advances, 7(33). https://doi.org/10.1126/sciadv.abe5641

Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press.

Cyberspace Administration of China. (2022, January 4). Provisions on the administration of algorithmic recommendation services in internet information services [in Chinese]. https://www.cac.gov.cn/2022-01/04/c_1642894606364259.htm?utm_source

Cyberspace Administration of China. (2024, June 15). Expert interpretation: Grasping the foundations and key priorities of online violence governance [in Chinese]. https://www.cac.gov.cn/2024-06/15/c_1720136503463025.htm

Guan, T., & Chen, X. (2025). Threat Perception, Otherness and Hate Speech in China’s Cyberspace. Journal of Contemporary China, 1–16. https://doi.org/10.1080/10670564.2025.2475051

Just, N., & Latzer, M. (2016). Governance by algorithms: reality construction by algorithmic selection on the Internet. Media, Culture & Society, 39(2), 238–258.

Liu, S. X. (2025, May 26). The Digital Ecology of Hate: Technology, Policy and Online Fields. Centre for International Governance Innovation. https://www.cigionline.org/publications/the-digital-ecology-of-hate-technology-policy-and-online-fields/

Massanari, A. (2017). #Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3), 329–346. https://doi.org/10.1177/1461444815608807

Ministry of Public Security of the People’s Republic of China. (2025, September 18). Public security authorities advance the “Clean Internet 2025” campaign and crack down on major online crimes in accordance with the law: 10 representative cases released by the Ministry of Public Security [in Chinese]. Ministry of Public Security of the People’s Republic of China. https://www.mps.gov.cn/n2254098/n4904352/c10237151/content.html

Morales, E., Hodson, J., O’Meara, V., Gruzd, A., & Mai, P. (2025). Online toxic speech as positioning acts: Hate as discursive mechanisms for othering and belonging. New Media & Society. https://doi.org/10.1177/14614448251338493

Su, S. (2026, April 11). Protect Quan Hongchan, but do not cyberbully Chen Yuxi [in Chinese]. Xinjing Bao [the Beijing News]. https://baijiahao.baidu.com/s?id=1862141648901069057&wfr=spider&for=pc

Xinhua News Agency. (2025, May 4). Diving—World Cup Super Final: Chen Yuxi and Quan Hongchan claim gold and silver in women’s 10m platform [in Chinese]. Xinhua Net. https://www.news.cn/sports/20250504/f987e5a8555b4ec6ace3065b08e23ead/c.html

Yulezibenlun. (2024, August 7). Sports fandomization: Are brands using it while also complaining about it? [in Chinese]. 36kr.com. https://36kr.com/p/2894966548732551