Responding to a sharp rise in AI-generated image-based abuse in Australia, recent reporting by ABC News has highlighted how women and girls are increasingly being targeted through deepfake technology — digitally altered or entirely fabricated explicit images created without consent.
As online safety experts warn, “once this kind of content is shared, it can be almost impossible to fully remove,” leaving victims with lasting emotional and reputational harm.”
This case shows that the transmission speed of social media is very fast. As soon as the content is shared, it will be constantly spread and cover most users, which is also terrible. Regulatory authorities such as the Australian eSafety Commissioner(2023) believes that the abuse of AI and portraits has caused serious harm to our lives, especially teenagers and female users. Therefore, how to change this phenomenon has become particularly important.
This case is not only a few phenomena, but also a common problem. Most people believe that hate speech is free speech from individual users, but this view is too one-sided. If we think more deeply, the platform also plays a crucial role. The repeated abuse of AI on social media has actually increased the attention of the platform. People’s likes, comments and retweets have increased the popularity of the topic, giving the platform and the company behind it a chance to make a profit, even if these behaviors hurt others.
This blog argues that the problem of harassment of women on the Internet is not only the bad behavior of individual users, but also a comprehensive and structural problem caused by platform governance, algorithm application and gender discrimination in a broad sense.
Why Women Are Mostly Affected
People often think that online harm is only the harassing remarks or malicious comments of individual users. But in fact, it’s not so single and simple. It is a complicated problem caused by many factors.
As Flew (2021) argues, “online harms are not simply matters of offensive content, but of the real-world consequences they produce.”
This shifts the focus from individual behavior to the actual impact of the platform. Especially in the operation mode of the platform, the risk level is different for each user. Compared with other users, women are more likely to be targeted by harmful remarks and harassment. Therefore, under such circumstances, women’s participation in social media has been weakened, and their rights have not been fully guaranteed.
This phenomenon reflects the harmful consequences of women under the framework of the platform’s promotion of free speech. At the same time, it also shows that the governance system of the platform seems reasonable and neutral, but in fact it is not unfair to women, and even the algorithm of the platform will aggravate this unfair and harmful behavior.
As Sinpeng et al. (2021) emphasize, a purely content-based definition of hate speech affects how harm in practice many forms of gender-based harassment continue to consist of silence, intimidation and exclusion.
This statement further confirms that women are more likely to be harassed on social media, and these subtle hurtful remarks cannot be detected by the platform in time. This shows that the governance of the platform has limitations, which may be technical limitations or limitations of traditional concepts. Some platforms with hints or harassment cannot be detected in time, which reflects the limited policies of the platform.
The platform’s definition of speech harm is too narrow so the seemingly fair rules exacerbate the harm of free speech to female users, which is a fundamental failure. Therefore, the fact that women are more vulnerable to gender-based harm is not a separate personal problem, but a structural phenomenon formed by the accumulation of the platform.
How Platform Decides
The rules and operation mode of the platform not only determine what content can be accepted, but also affect who will be hurt. In 2025, Meta changed its policy to ” more speech, fewer mistakes,” to reduce moderation errors and increase content on the platform. This seems to support free speech, but in fact it leads to a deeper question: how does the platform work?(2025)
As Flew (2021) suggests, regulating online communication involves balancing competing values, particularly between protecting speech and preventing harm.
This change seems to be beneficial to free speech, but it has also raised concerns among mass users about platform governance.
It is not easy to regulate and balance the relationship between the two, because it involves two important values: protecting speech and preventing harm. In a sense, the two cannot be well unified, especially for women who are already vulnerable to malicious harm on the Internet. This platform governance method that promotes free speech will only make them see more harmful remarks and be hurt by them. Based on this, we should further strengthen the governance of free speech and online harm to achieve a balance between them.
The example of Meta also shows that platform governance is not neutral. It can not only standardize the content of the publication, but also create conditions to cause harm. When prioritizing content and comments, the platform may inadvertently intensify the spread of harmful content, especially real-time events or high-attention events, which makes female users more susceptible to influence.
The platform claims to be neutral, but it is designed to maximize topic discussion and user participation, even if these contents are harmful remarks. This contradicts its self-pret, so the platform is not neutral.
Therefore, due to this unequal governance system, the regulatory focus of the platform should shift from eliminating hazards to risks that users can accept.
What the Data in Australia Shows
Australian data from eSafety Commissioner(2023) shows: Women are more likely than men to be the target of sexual and gendered abuse that happens online or uses digital technology. With about one in three women reporting, having experienced work-related online abuse.
These findings are important not only because of their large scale, but also because cyber abuse shows a specific pattern: cyber abuse does not affect everyone equally, but has a targeted impact on socially marginalized groups and gender discrimination. This abuse is closely linked to two factors:
- Gender discrimination: based on harmful attitudes, beliefs, stereotypes and behaviors related to gender
- Gender inequality: caused by power imbalances in society and unequal treatment based on gender
More importantly, these patterns point to a deeper issue. If certain groups are consistently more affected, then the problem cannot be explained by individual behavior alone. Instead, it suggests that platform environments shape who is more exposed to harm.
From this perspective, the data does more than show the scale of the issue, it reveals how inequality is reproduced through digital systems.
Algorithm Behind The Platforms
The algorithm behind the social platform will amplify the content of the interaction, even if these interactions are harmful. For example, harmful remarks or defamatory language has triggered intense public discussion, improved visibility and user participation, and frequently pushed to make these harmful content appear in the public vision.
Visibility in social media is not neutral, it is characterized by algorithmic systems that prioritize interactions. Those content that triggers strong reactions, even if they are harmful, such as harassment, are more likely to be amplified.
This means that content is not simply shared but treated with preference. Posts that elicits strong reactions, whether positive or negative, are more likely to be redistributed.
Massanari (2017) demonstrates how the design of platforms creates a toxic environment and amplifies harassment, rather than merely allowing it.
This suggests that online abuse is not only tolerated, but indirectly encouraged. When harmful content generates clicks, shares, and comments, platforms have little financial incentive to remove it.
This creates a feedback loop:
- Harmful content generates engagement
- Engagement increases visibility
- Increased visibility normalises the behavior
Platform design not only creates a breeding ground for harmful activities, but actively amplifies and normalizes them.
This dynamic cycle is especially obvious in gender-based harassment. These offensive remarks are easy to cause discussion and controversy among users, thus affecting the push of the content by the platform algorithm. Under such circumstances, the spread of harmful speech is not because of the value of the content itself, but because they effectively attract the interest and attention of users, thus forming a cycle.
This makes harmful content structurally more visible, while harmless and factual content is relatively marginalized.
In short, the platform design does not really reflect the behavior of users, but promotes the spread of hot topics, so women are still vulnerable to harassment under the platform algorithm.
New Technologies, New Risks
New technologies make online harm more complex and difficult to control. Deepfake technology enables the creation of incredibly realistic but completely fake images that are often misused to avoid consensual, producing obscene content directed at women.
As recent cases in Australia show, such material can be created and shared in no time, even within school communities What is particularly dangerous about this type of abuse is its persistence and scale once shared, images can be difficult to completely remove They keep popping up on different platforms and causing ongoing emotional distress and reputational damage.
This marks a change in the nature of online harm malicious acts are no longer limited to user-generated comments, but increasingly include AI-generated content.
More importantly, the nature of harm itself has changed It is no longer about “what people say”, but “what technology can produce”. As these tools become more widespread, existing moderation systems could reach their limits.
At the same time, this harm does not spread by itself. Platform systems and algorithms additionally amplify this content, especially when it attracts attention. This is because the platform amplifies the content, not just because of the existence of technology.
Rethinking Responsibility
If online harassment is shaped by platforms, the responsibility does not lie with users alone. Currently, people are often encouraged to block, report or ignore harmful content. This may be helpful in the short term, but does not solve the underlying problem-why this type of content spreads so easily.
Platforms are designed to generate maximum attention. Algorithms try to spread content that elicits reactions, often anger and harassment at the same time, governance decisions such as Meta ‘s move towards more freedom of expression help keep harmful content visible.
Together with these algorithms and systems, they create a network environment where harmful speech will spread quickly. The core problem is that if the dissemination and interaction of these harmful contents can bring benefits to the platform, the platform won’ have motivation to stop it. In other words, harmful behavior on social platforms is not only a problem of bad users, but also a problem of the platform system itself. And this kind of temptation makes it difficult for the platform to control.
Therefore, it means a rethinking of responsibility, shifting the focus from individuals to platforms-including designing algorithms, setting rules and structuring online spaces. It remains a limited conclusion as long as these deeper problems are not addressed.
Conclusion: Beyond Individual Behavior
Online harm is often understood as individual behavior – the irresponsible remarks of a few bad users. And this explanation is too simple. There are more and deeper reasons behind the network damage.
From the example of deep AI forgery, to the transformation of Meta’s launch of more speech, to the data on women’s network harm in Australia, putting these together, we can conclude the fixed operation mode of the platform: the damage is not random, but jointly influenced by the algorithm and structure of the platform. Deepfake technology has increased the generation of harmful content. Governance decision-making determines which content can be disseminated, and the algorithm determines which content can be seen and shared.
These factors do not work alone, but work together to form a structured platform that attracts users’ attention and visibility. This gradually normalizes the spread of hurtful remarks on social platforms. Therefore, some users are more at risk than others, especially women, who are vulnerable to gender discrimination or harassment. This is not a personal behavior, but the role of the platform, which promotes such harm.
This phenomenon shows that the so-called neutral platform is not completely neutral. The policy promises to strike a balance between free speech and preventing harm, but this is not the case. He is unequal. When relaxed management and algorithms prioritize interaction, risks will not be evenly distributed. Female users and some vulnerable groups will become the most vulnerable part, and the responsibility will still be attributed to individual users.
Users are expected to report harmful content and harmful remarks, but this does not fundamentally solve the harm caused by harmful remarks to women. Therefore, this is not the responsibility of a single user but requires the platform to make efforts. The platform should rethink its operation mode and governance method, which may be to control harmful content more accurately and in a timely manner, or improve the efficiency and level of algorithm operation, and set restrictions on the dissemination of harmful content rather than encouring.
However, only when the platform truly turns the monitoring of individual behavior to reflection on its own systems and technologies can we see a breakthrough in more meaningful and substantive solutions. This requires seeing the needs, risks and harm suffered by users. Only when network damage is regarded as a structural problem rather than a personal problem can the platform make better progress and achieve a harmonious online social relationship.
Reference
eSafety Commissioner. (2023). Women and online safety. https://www.esafety.gov.au/women/reduce-technology-facilitated-abuse
Flew, Terry (2021) Hate Speech and Online Abuse. In Regulating Platforms. Cambridge: Polity, pp. 91-96 (pp. 115-118 in some digital versions
Massanari, Adrienne (2017) #Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3): 329–346.
Meta. (2025). More speech, fewer mistakes. https://about.fb.com/news/2025/01/meta-more-speech-fewer-mistakes/
Sinpeng, A., Martin, F., Gelber, K., & Shields, K. (2021, July 5). Facebook: Regulating hate speech in the Asia Pacific. Final Report to Facebook under the auspices of its Content Policy Research on Social Media Platforms Award. Dept of Media and Communication, University of Sydney and School of Political Science and International Studies, University of Queensland. https://r2pasiapacific.org/files/7099/2021_Facebook_hate_speech_Asia_report.pdf
Be the first to comment