One day, after a good night’s sleep, you open your phone only to discover that racy photos of you are spreading online. But these photos aren’t real, and you’ve never taken them. Your face has been digitally swapped onto someone else’s body, yet people on the other side of the screen believe them to be genuine.
This isn’t a fantasy. It’s exactly what happened to Taylor Swift in 2024.
In early 2024, deepfake pornography of Taylor Swift went wildly viral on X (Twitter). When the platform took action, these images had been viewed and shared millions of times by users, and some have even been saved to today.
This incident has exposed numerous shortcomings and issues regarding the application and regulation of deepfakes.
In this blog, I think that deepfake pornography is not merely a case of users misusing AI. More importantly, it is a matter of platform and state governance. The fact that this incident happened was caused by a combination of factors, including the platform and regulatory laws.
What Is a Deepfake
A “deepfake” is a type of media created using AI. It can change videos, images, or even audio to look very real. Most deepfakes are made using a system called Generative Adversarial Networks (GANs).
Figure 1. How deepfake works. Source: ResearchGat
It can easily swap the faces of two people, or make one’s own face perform someone else’s expressions. This technology is not new, it has long been used effectively in the film and entertainment industries. However, in recent years, many people have been using this technology to cause harm, particularly to women.
A 2023 Report showed that deepfake pornography made up 98 percent of all deepfake videos online, and 99 percent depicted women. Deepfake videos were an estimated 550 per cent more prevalent in 2023 than in 2019, and the tools to create them are widely available, usually free, and require very little technical expertise.（Security Hero, 2023）
The figures in this report are terrifying. Are they really that high? Let’s set that aside for now. What is certain, however, is that the very first deepfake image involved face-swapping on a woman, and at the height of the technological boom, many Hollywood actresses fell victim to face-swapping. Indeed, the majority of victims are women. Although these images are fake, the harm caused to the victims is very real.
A recent study (Panorama Global, 2023) indicates that more than half of deepfake victims in the US have suicidal thoughts. As we know, once these images are uploaded, it is virtually impossible to delete them completely, as they can be endlessly replicated and shared.
A simple explanation about how deepfake works:
Case Study: The Taylor Swift Deepfake Incident
Overnight, deepfake pornographic images of Taylor Swift spread rapidly across the X platform. One particularly extreme image remained on the platform for 17 hours, causing great outrage amongst her fans. It garnered 47 million views and 240,000 likes before the platform finally blocked the account that had posted it. However, many people had already saved the image, meaning this form of online harm endured.
Ironically, the ban was not due to the platform considering the image to be harmful, but rather as a result of huge pressure, such as mass reports from fans and significant public criticism.
Her fans are incredibly effective. They started the #ProtectTaylorSwift movement, uploading a huge number of beautiful videos and photos of Taylor Swift using the same hashtag. They wanted to use this method to resist the deepfake pornographic content.
In contrast to these proactive fans, the multi-billion-dollar platform took no action for 17 hours. Only under huge pressure did they reluctantly take emergency measures, which amounted to a complete ban on searches for the keyword ‘Taylor Swift’.
Even after this, the user who deliberately posted this content was not punished. Is this a failure of the platform, or a gap in its laws? Maybe both.
Why Platforms Failed
Social media has been around for many years, and the technology behind these platforms is now mature, with targeted content recommendation systems in place. Yet why is it not possible to prevent the spread of such deepfake pornography? If such content could be controlled before it is disseminated, the number of victims would be much lower.
To find the answer to this question, we must first understand the nature of platform governance.
Today, platforms are no longer mere intermediaries but act as de facto regulators of online speech (Flew, 2021). Platforms determine which content is displayed, which is removed, and which is promoted. So, what are the principles guiding these decisions?
Firstly, these platforms are private businesses whose first aim is to earn profits and secure a stronger market position. They may have to balance profitability against the public interest, but will they put their own profits first or the rights of individual users?
It is clear from the way they have handled so many incidents involving deepfake pornography—which they have generally dealt with poorly—that they have chosen to put their own profits first.
For example, in content selection, they prioritize content with higher traffic, maximizing views and engagement to generate more profit, rather than considering whether the content constitutes gender-based violence or other harmful effects, or the extent of harm it may cause to others.
Regarding content moderation, most platforms rely on user reports and automated detection systems, significantly reducing moderation costs. However, this approach results in inefficient governance and an inability to control the situation before public opinion escalates.
Furthermore, many have observed that platforms only act quickly when the victim is a high-profile celebrity. In contrast, platforms frequently ignore ordinary women harmed by nonconsensual deepfake content. In this way, platforms have a lot of power. But this power is driven more by business interests than by the public good.
Figure 3. The response to the Taylor Swift deepfake incident. Source: Screenshot from X (Twitter)
Returning to the Taylor Swift incident: the viral spread of deepfake pornography within 17 hours generated enormous profits for platforms, which only took belated action under intense public and fan pressure.
If the victim had been an ordinary woman, even if the content reached a smaller audience, platforms would most likely ignore the case entirely, disregarding her rights, dignity, and well-being.
How Algorithms Made It Worse
To be honest, we see all kinds of content on social media every day, and there are some things that simply cannot go wide no matter how much we might want them to. But why is it that deepfake pornographic content always seems to spread so quickly?
Research suggests that pornographic content is more likely to attract public attention and achieve higher engagement. The higher the engagement, the more the algorithm detects that many people enjoy viewing such content.
The algorithm cannot judge whether the consequences of this preference are positive or negative. It simply increases the promotion of content that people enjoy viewing, like shocking, emotional, or sexual.
Figure 4. How algorithms amplify harmful content. Source: author’s own illustration
Let us now consider this issue from a gender standpoint. In our experience, we have indeed seen many female celebrities fall victim to deepfake pornography, so why do we rarely see deepfake pornography involving men? This is, in fact, a long-standing issue of gender inequality.
Women and their bodies have long been commodified online. For the greater part of the mid-20th century, traditional advertising media, like perfumes, colognes, clothing, underwear, and even food, used the female body in a sexualised manner. (Frey, 2025) And this is just one of the root causes of these harmful consequences. Algorithms, too, have unknowingly worsened this commodification.
As Massanari (2017) demonstrates in her analysis of Reddit during Gamergate and “The Fappening,” platform architectures and governance structures can actively enable what she terms “toxic technocultures.” Reddit’s upvote-based visibility system rewarded controversial and sexualized content, embedding gendered harm within platform infrastructures.
Why the Law Is Falling Behind
As society evolves, an increasing number of new technologies are emerging. However, the laws governing these technologies have not kept pace, allowing individuals to publish deepfake pornographic content that causes significant harm to others. Yet they continue to go unpunished.
In the United States, there is currently no uniform federal legislation governing deepfake technology (Graham, 2024). Although individual states have established their own laws, this fragmented legal system proves almost ineffective in regulating online content, as such content crosses national boundaries.
It is not just the United States. Most countries lack specific laws targeting deepfake pornography. This allows content creators to act with even greater boldness.
Even if every nation were to enact specific laws against deepfake pornography, cross-border enforcement would present a major challenge.
Nowadays, many social media platforms operate across national borders. Deepfake content may be created in one country, uploaded in another, and hosted on a platform registered in a third.
This scenario may seem complex, but it is all too common. That is why we find it so difficult to make laws, and even harder to enforce them.
In addition to the challenge of cross-border governance, every nation must also consider the issue of the extent of regulation. Too much regulation can be harmful.
For example, it may threaten freedom of speech and spark new public complaints. So policymakers need to get the balance right between effective regulation and ensuring people’s rights.
So， What Can Be Done?
Facing these various difficulties, we cannot simply require platforms to step up their content moderation, nor can we ask every nation to improve its own laws and engage in cooperation. No single entity can resolve such a global issue on its own.
Just as Nicholas Suzor (2019) described the need for “digital constitutionalism,” power in cyberspace should be constrained by common norms, accountability mechanisms, and public oversight.
What is truly needed is a combined effort from all hands. Only through cooperation between platforms, governments, law enforcement agencies and the wider online user community will it be possible to gradually resolve this issue.
1. For platforms: Platforms cannot focus purely on their own profits and market position. They must make greater efforts to consider the public interests. Putting the public interests first doesn’t mean they won’t make a profit. On the contrary, doing so will earn more support and trust from users. This is also a step in the right direction.
At the same time, they must increase costs for content moderation. They need to design a more flexible and responsive moderation system to stop the spread of this deepfake pornographic content before it starts.
As Tarleton Gillespie (2018) argues, Content moderation is a form of governance that shapes public discourse, not merely a technological process. Therefore, platforms also need to increase the transparency of their algorithms, allowing the public to question and propose improvements to the algorithmic mechanisms.
2. For states: Countries need to update their laws timely to ensure that criminals are punished promptly. Also, they must strengthen cooperation with other countries. Several countries can work together to draft relevant laws and discuss common international action plans. This will enable them to overcome national boundaries and address issues more effectively.
Furthermore, all countries should have support services in place for those who have already been harmed. Such as free legal assistance, counselling and so on.
3. For people: we should learn to protect ourselves. If something happens that harms our interests, we must actively seek help and have the courage to fight back. Girls, in particular, must say no to sexual harm.
Figure 5. A beautiful image of Taylor Swift. Source: ABC News
Conclusion
Many people think the spread of deepfake pornography is a technical issue. However, I believe it is a matter of governance involving multiple entities. The Taylor Swift incident shows that platforms’ content moderation and algorithms can directly amplify the impact and harm caused by such incidents. Meanwhile, the delays in national laws allow more criminals to escape justice.
This shows we cannot rely on a single entity to completely resolve the issue of the spread of deepfake pornographic content. The most effective way is for multiple entities to work together on governance and regulation. They must clearly understand their responsibilities and actively fulfill them.
Finally, we feel great sympathy for those harmed by deepfake pornography, while also feeling anger at the platforms and the law. But if we truly want to change this situation, we must work together!
References
Flew, T. (2021). Regulating platforms. Polity Press.
Frey, H. (2025). The commercialization of bodies: How industries commodify womanhood. HASTAC. https://social-political-issues.hastac.hcommons.org/2025/12/13/the-com mercialization-of-bodies-how-industries-commodify-womanhood/
Gillespie, T. (2018). Custodians of the internet: Platforms, content moderation, and the hidden decisions that shape social media. Yale University Press.
Graham, M. M. (2024). Deepfakes: Federal and state regulation aims to curb a growing threat. Thomson Reuters. https://www.thomsonreuters.com/en-us/posts/government/deepfakes-fe deral-state-regulation/
Massanari, A. (2017). #Gamergate and the fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3), 329–346. https://doi.org/10.1177/1461444815608807
Panorama Global. (2023). I didn’t consent: A global landscape report on image-based sexual abuse. https://panoramaglobal.org/knowledge-center/articles/image-based-abu se-initiative-landscape-report-2023/
Security Hero. (2023). The state of deepfakes 2023. https://www.securityhero.io/state-of-deepfakes/
Suzor, N. (2019). Lawless: The secret rules that govern our digital lives. Cambridge University Press.
UN Women. (2026). Deepfake technology and violence against women. https://www.unwomen.org
Be the first to comment