Social media connecting people is something we have always thought about. The next era of social media – the era of AI – is coming, but things are still “very chaotic.” In today’s digital world, harmful comments, abuse, and misogynistic hate can travel fast, and often with few real consequences.
The result is bigger than a moderation failure: these platforms are not just failing to stop gendered hate, but are actively creating the conditions that allow it to thrive.
The Numbers Behind Online Sexism
Hate speech has been increasingly used as a weapon of gender-based violence meant to intimidate women into silence.
A 2016 study found that 76% of women under 30 years of age, have reported experiencing online harassment, and almost half (47%) of all women had been targets. More recent research on the experiences of women in Australia found that, of those that had experienced online abuse and harassment, 42% of women said it was misogynistic or sexist in nature, and 20% said it had included threats of physical or sexual violence.
When considering the above-mentioned figures, widespread automation is anticipated to have different social and economic effects on men and women, which could have a big impact on worldwide gender inequality and socioeconomic equality.
Online hate speech is a global issue that has become even more visible in the age of Web 2.0. Platforms like TikTok, Instagram, Reddit, and Facebook act as major cultural intermediaries. They have the ability to shape how people think, interact, and understand the world around them. When women are targeted online, the harm rarely stays on screen. It can affect their mental health, damage their professional opportunities, and push them out of digital spaces altogether.
What users see online, and just as importantly, what gets ignored or allowed to spread, is shaped by the rules platforms set and the algorithms they choose to prioritize. Although the regulations applied to this issue vary across countries, generally, a structural mismatch between platform governance, algorithmic amplification, and cultural norms is the main factor causing sexism on social media to persist. Despite moderation efforts, the system itself still makes it far too easy for gendered hate to spread.
What Counts as Sexist Hate Speech?
The definition of hate speech is used in many different ways. However, it should be used to denote a type of speech that is sufficiently harmful to be regulated, in contrast to types of insulting or offensive speech that should not. Or as Parekh (2012) defined: speech that ‘expresses, encourages, stirs up, or incites hatred against a group of individuals distinguished by a particular feature or set of features such as race, ethnicity, gender, religion, nationalist, or sexual orientation’. The speech itself need not necessarily be violent or emotive, or to lead to public violence in and of itself to be hate speech.
International Media Support provided a more detailed definition: Sexist hate speech relates to expressions which spread, incite, promote or justify hatred based specifically on a person’s sex or gender.
In the era of AI, sexist hate speech is also expressed through the form of ‘deepfakes‘ – understood as highly realistic but fake digital material that has been created using artificial intelligence and which depicts a real person doing or saying something that they did not do or say. Sexually explicit deepfakes, or deepfake pornography, accounted for 98% of the 95,820 deepfake videos that were online in 2023, of which, 99% of the persons depicted in sexually explicit deepfakes were women.
As Sinpeng et al. (2021) point out, platforms are far more likely to act on obvious abuse, like direct threats, than the quieter, everyday sexism that women deal with online. This creates enforcement gaps where sexist content can circulate freely without violating platform regulations. For instance, stereotypes about women, such as roles in the family and body image, are often not classified as hate speech by platforms, even though these elements reinforce sexism and exacerbate the online environment. This phenomenon can be considered as an “escape” from the censorship of social media platforms.
Regardless of the definitions mentioned above, it is how each platform defines hate speech, or more specifically, sexist hate speech, that shapes the response to this issue.
In this sense, the problem does not seem to lie in how regulations are enacted, but rather in the limitations of how sexist hate speech is defined and recognized within platform governance systems.
Can Platforms Really Protect Women?
As Flew (2021) argues, platforms function as private regulators: they define acceptable speech, enforce rules, and shape visibility.
In the network of our communication, cyberspace is made up of relationships, transactions, and ideas that ripple through it like a standing wave. However, it appears that hate and misogyny are increasingly being spread via cyberspace.
Social media platforms increasingly function as regulators of public discourse. However, all these regulations share common problems: they are inconsistent, they only prioritize profit, and they are decisions made only after problems arise.
For instance, the incident involving AI-generated content created without user consent has raised concerns about platform X’s indecisive action. Ele is a content creator who posts non-nude sexual content on X and on paid subscription sites. However, Ele’s image was ‘undressed’ by Grok, an AI bot language model that has been promoted by X, at the request of other users.
Despite repeated concerns about the harmful content users have reported from Grok, X continues to promote the tool, largely because of the commercial value and user engagement it brings to the platform.
The only response to this issue is a post by Elon Musk, who wrote that anyone who used Grok to create illegal content would be punished. But since Musk’s comment, the bot has continued to respond by prompting requests to undress women, and in some cases, children.
Algorithms Reward Misogyny
If platforms create regulations and attempt to redefine hate speech to create a more positive online environment, then the algorithms of these platforms are also a factor that allows sexism to thrive online. As Massanari (2017) suggests, platform architectures and users work together to produce “toxic technocultures,” where harmful content is not just permitted but amplified. When content or comments “escaped” from the censorship, they circulate through algorithmic recommendation systems designed to maximize engagement.
Platforms like TikTok, YouTube, Facebook, and X have algorithms that prioritize likes, shares, comments, and clicks rather than social harm. Viral content appearing on these platforms may not reflect what users are interested in, but simply provoke anger, prompting users to react or engage in indignant responses.
Real-world cases clearly demonstrate this. Dr. Kaitlyn Regehr (UCL Information Studies) conducted research to show that the TikTok algorithm was presenting four times as many videos with misogynistic content, including objectification, sexual harassment, or discrediting women, to users seeking content on masculinity or loneliness. These systems do not actively promote sexism, but instead push this content to people who are likely to express opinions on these issues.
Taylor Swift Deepfake Scandal: How X Helped AI-Generated Sexist Abuse Go Viral
In January 2024, non-consensual and sexually explicit images of Taylor Swift were widely circulated on X (formerly Twitter) and other platforms. An image shared by a user on this platform was viewed 47 million times before the account was suspended. X attempted to suspend several accounts that posted the faked images of Taylor Swift, but the images were shared on other social media platforms and continued to spread despite those companies’ efforts to remove them. The images spread so quickly that even users who were not searching for them were exposed through recommendations and repost networks.
The most viewed and shared deepfakes of Swift portrayed her nude in a football stadium. Taylor Swift has faced sexist hate speech for a long time, especially since publicly supporting her partner, Kansas City Chiefs player Travis Kelce, by attending NFL games.
In fact, this incident is not the first time X has reacted slowly or failed to address the issue of sexually explicit deepfakes on the platform. Previously, the platform was also slow to remove sexually explicit deepfakes of a 17-year-old Marvel star or TikTok stars circulating on the platform.
Like other platforms, X’s algorithm allows this kind of unethical content to spread. The more retweets, likes, quotes, and trending topics about this story, the faster this sexist content will spread across users’ feeds. This is a clear example of “toxic technocultures“: when platform features, algorithms, and even users unintentionally and intentionally normalize misogyny, they all contribute to the spread of the sexist narrative online.
Despite numerous efforts of platforms to address sexist content like this, people still find ways to break the rules. “It’s an arms race, and it seems that whenever somebody comes up with a guardrail, someone else figures out how to jailbreak,” said Oren Etzioni, a computer science professor at the University of Washington who works on deepfake detection. It is worth mentioning that tech platforms like X, which have developed Grok as their own generative-AI products, have yet to deploy or discuss tools to detect generative-AI content that goes against their guidelines on sexism or hate speech.
As Coleman (2013) points out in her ethnographic work, the “leaderless” nature of online movements makes them a cover for individual selfishness and competitiveness. It begins with each person’s perspective, spreads through the platform’s algorithms, is tolerated by lax regulations, and can only end when few people care about it.
Why Current Solutions Fail
The core issue is that platforms are built to maximize engagement and hold users’ attention for as long as possible. The rise of artificial intelligence (AI) in content automation has strengthened this model, regardless of ethical concerns or environmental harm.
Flew (2021) discussed the growing power of platforms as private regulators of public discourse; however, all of this lacked corresponding accountability. Sexism hate speech is structurally enabled, not because of a lack of regulation from the platforms. Through the case studies mentioned above, whether through belated reactions or even complete ignoring of the issue, platforms still prioritize their economic interests.
In reality, the different definitions and handling of hate speech across regions have made it difficult for platform-specific regulations to be effective. For instance, as Sinpeng (2021) mentioned, hate speech legislation across the Asia Pacific is a mixed and varied array of rules, or proposed rules, based in some measure on constitutional law, but also reliant on penal codes and civil laws.
Many countries have criminal laws relating to hate speech, including Canada and Germany, which survive constitutional free-speech challenges because they are regarded as appropriate and proportionate to the achievement of other constitutional values, such as protection from the harm of discrimination or the preservation of human dignity.
What Must Change
Cyberspace governance is a major challenge of the 21st century. However, based on observations, several changes are needed to gradually address online sexism.
A unified definition of online sexism and hate speech needs to be established. As Sinpeng et al. (2021) mentioned, “Facebook’s definition of hate speech is narrower than international human rights standards”, hate speech will continue if platforms do not agree on what behavior is considered a violation of ethics and cyberspace standards.
Sinpeng et al (2021) also proposed the role of page administrators as critical gatekeepers of hate speech content and support their improved regulatory literacy through training and education. In this context, the responsibility for providing mandatory hate speech moderation training modules should lie with parent companies, so their local branches can adapt these standards and comply with national laws.
The most important thing each individual needs to understand is that when human rights become a topic of discussion, platforms are no longer neutral intermediaries; they need to play a stronger role in shaping user behavior. For this reason, creating a safe online environment is a shared responsibility of governments, platforms, and users alike.
References
Australian Human Rights Commission. (2018, July 24). Human rights and technology issues paper. https://tech.humanrights.gov.au/sites/default/files/2018-07/Human%20Rights%20and%20Technology%20Issues%20Paper%20FINAL.pdf
Barlow, J. P. (1996, February 8). A Declaration of the Independence of Cyberspace. Electronic Frontier Foundation. https://www.eff.org/cyberspace-independence
Cappadona, B. (2023, December 6). Taylor Swift Opens Up About Travis Kelce in Time’s Person of the Year Story. TODAY.com; TODAY. https://www.today.com/popculture/taylor-swift-travis-kelce-time-magazine-rcna128259
CBS News. (2023, December 4). Deepfake nude images of teen girls prompt action from parents, lawmakers: “AI pandemic”. https://www.cbsnews.com/news/deepfake-nude-images-teen-girls-action-parents-lawmakers-ai-pandemic/
Down, A. (2026, March 31). “Women who speak out must be exterminated”: the rising tide of digital violence facing Ethiopian activists. The Guardian; The Guardian. https://www.theguardian.com/global-development/2026/mar/31/digital-violence-facing-ethiopian-women-activists-online
D’Souza, Tanya; Griffin, Laura; Shackleton, Nicole; Walt, Danielle – “Harming Women with Words: The Failure of Australian Law to Prohibit Gendered Hate Speech” [2018] UNSWLawJl 32; (2018) 41(3) UNSW Law Journal 939. (2018). Austlii.edu.au. https://www5.austlii.edu.au/au/journals/UNSWLawJl/2018/32.html
Fabrizio Germano, Vicenç Gómez, Francesco Sobbrio, Ranking for engagement: How social media algorithms fuel misinformation and polarization, Journal of Public Economics, Volume 255, 2026, 105589, ISSN 0047-2727, https://doi.org/10.1016/j.jpubeco.2026.105589
G.A. Res. 2106 (XX), at 48, U.N. Doc. A/RES/2106 A (XX) (Dec. 21, 1965); G.A. Res. 2200A (XXI), at 55, U.N. Doc. A/RES/2200A (XXI) (Dec. 16, 1966); Simon Bronitt, “Hate Speech, Sedition and the War on Terror,” in Hate Speech and Freedom of Speech in Australia 135–6 (Katharine Gelber & Adrienne Stone eds., Federation Press 2007) [hereinafter Hate Speech in Australia].
Graham, T. (2024, November 6). Elon Musk’s flood of US election tweets may look chaotic. My data reveals an alarming strategy. The Conversation. https://theconversation.com/elon-musks-flood-of-us-election-tweets-may-look-chaotic-my-data-reveals-an-alarming-strategy-243021
H Ajder, G Patrini, F Cavalli & L Cullen, Deeptrace Labs. (2019). The state of deepfakes: Landscape, threats, and impact. https://regmedia.co.uk/2019/10/08/deepfake_report.pdf
Herz M, Molnar P, eds. The Content and Context of Hate Speech: Rethinking Regulation and Responses. Cambridge University Press; 2012.
International Media Support. (2023). Online gendered disinformation and sexist hate speech (Learning brief). https://www.mediasupport.org/wp-content/uploads/2023/03/IMS-Online-gendered-disinformation_final.pdf
Issues of Concern. (2021). In T. Flew, Regulating platforms (pp. 91–96). Polity.
Law, T. (2017, January 4). Hate Speech Against Women in African Media – Ethics in the News. Ethical Journalism Network. https://ethicaljournalismnetwork.org/africa
Massanari, A. (2017). Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3), 329–346. https://doi.org/10.1177/1461444815608807
New South Wales Parliamentary Research Service. (2024). Sexually explicit deepfakes. https://www.parliament.nsw.gov.au/researchpapers/Documents/Sexually%20explicit%20deepfakes.pdf
Security Hero. (2023). The state of deepfakes. https://www.securityhero.io/state-of-deepfakes/
Sinpeng, A., Martin, F. R., Gelber, K., & Shields, K. (2021). Facebook: Regulating Hate Speech in the Asia Pacific. Department of Media and Communications, The University of Sydney. https://hdl.handle.net/2123/25116.3
Smith, L., & Niker, F. (2021). What Social Media Facilitates, Social Media should Regulate: Duties in the New Public Sphere. The Political Quarterly, 92(4). https://doi.org/10.1111/1467-923x.13011
Tenbarge, K. (2024, January 25). Nude deepfakes of Taylor Swift went viral on X, evading moderation and sparking outrage. NBC News. https://www.nbcnews.com/tech/misinformation/taylor-swift-nude-deepfake-goes-viral-x-platform-rules-rcna135669
Weale, S. (2024, February 6). Social media algorithms “amplifying misogynistic content.” The Guardian. https://www.theguardian.com/media/2024/feb/06/social-media-algorithms-amplifying-misogynistic-content
Be the first to comment