Think about the last photo you posted online. Maybe it was a selfie, a birthday shot, or just a casual picture with friends. Now imagine that photo being used without your knowledge, to put your face into a video you never agreed to be part of. That video could be funny and harmless. Or it could be explicit, humiliating, and completely out of your control.
And this isn’t just hypothetical anymore. This is already happening.
Deepfakes are AI-generated videos or images that can swap or manipulate faces in a realistic way (Department of Homeland Security, 2023). These systems work by analysing large datasets of facial images to learn how to imitate expressions, movements, and lighting with striking realism (Westerlund, 2019). As these tools become cheaper, faster, and easier to use, they have moved from niche use into everyday digital platfroms.
The technology itself is not the problem. A knife can prepare a meal or cause harm. The same logic applies to deepfakes. They can be used for creative storytelling, cultural preservation, or entertainment. What makes this difficult is not the existence of the technology, but the fact that we cannot control how everyone uses it. At the same time, the systems that are meant to shape its use are not keeping up. As a result, laws, platforms, and regulators have consistently fallen behind.
This is why deepfakes are not just a technology problem, but a governance problem. The real question is not how to stop the technology, but how to prevent it from being used in harmful ways.
From Entertainment to Risk: When Deepfakes Become Normal
To understand the problem, it might be helpful to start with something that seems completely innocent.
Take JiMeng, a popular Chinese AI app developed by ByteDance. With just one photo and a text prompt, the app can generate a cinematic scene and place your face right into it — turning you into the lead of a period drama, a sci-fi blockbuster, or whatever world you can imagine.
The generated effects are high-quality, shareable, and often go viral quickly. People enjoy using them. These videos often get thousands of likes and quickly turn into trends.
And that is exactly where the story gets complicated.
When face-swapping becomes this easy, fun, and socially rewarded, we stop questioning what it actually means to use someone else’s face.
An important question starts to emerge: do we still own our own faces?
We start to assume that if the technology allows it, it must be fine. Therefore, consent becomes an afterthought, and boundaries begin to blur. The more we see faces being swapped in entertaining ways, the more it starts to feel like a perfectly normal thing to do.
This process of normalisation matters more than it might seem. As Crawford (2021) argues, AI is not a standalone technology but a product of existing political and social structures, ultimately shaped to serve those already in power. When a platform rewards face-swapping content with likes and shares, it is not just enabling entertainment. It is gradually training users to see other people’s faces as raw material, something that can be borrowed, remixed, and consumed without asking permission. Technology did not create this attitude. But the environment around it makes it much easier for people to act on it.
Once that mindset becomes widespread, it’s much easier for things to cross the line.
When Losing Your Face Means Losing Control
The most serious harm caused by deepfakes is non-consensual intimate imagery: explicit videos created using someone’s face without their permission (Department of Homeland Security, 2023).
In South Korea, this has become a large-scale crisis. Women’s faces, taken from social media profiles, group chats, and even university directories, have been inserted into sexually explicit videos and distributed through private channels (Mackenzie & Choi, 2024).
The victims include students, teachers, celebrities, and ordinary women who simply have an online presence. Most found out not through any official alert or legal notice, but because a friend or colleague happened to stumble across it.
What makes this kind of harm so difficult to address is its invisibility. Victims often have no idea where the content was created, who made it, or how far it has already spread. This opacity is not accidental; it reflects a deeper imbalance in how digital systems are designed. As Pasquale (2015) argues, digital systems operate like a one-way mirror, where corporations hold vast knowledge about individuals, while the algorithms making decisions that affect people’s lives remain completely hidden from public scrutiny.
For deepfake victims, this creates a double harm.
First, there is the violation of having your face weaponised without consent.
Second, it is almost impossible to track down and remove the content once it is out there.
It is also important to recognise that deepfakes do not affect everyone equally. Women are disproportionately targeted, and this phenomenon reflects broader social inequalities rather than anything inherent to the technology itself. As Crawford (2021) pointed out, AI systems tend to amplify the existing power structures in society.
The deepfake crisis in South Korea is not a random outcome, but a product of a specific social context. In this context, women’s bodies have long been subject to public scrutiny, and this technology makes image-based abuse cheaper and easier than ever before.
From Victims to Activists: The Law Is Playing Catch-Up, and Losing
So, where is the legal protection?
The short answer is: it is not keeping up.
In the United Kingdom, victims of deepfake abuse have begun to speak publicly about their experiences, partly to seek justice, and partly because seeking justice through existing legal channels has proven so difficult.
One victim, Jodie, described how images of her were used in explicit deepfake content by her friend. When she attempted to take legal action, she found the law offered her almost nothing (Moore, 2025). There was no specific offence that clearly applied. The person responsible faced no meaningful consequences.
The problem is structural. Many legal frameworks were designed around different kinds of harm like, physical assault, defamation, and theft of physical property. Deepfakes do not fit neatly into any of these categories. They exist in a legal grey zone where intent matters enormously. If someone claims they shared content as a joke, or if it circulated within a private group rather than publicly, existing laws often cannot reach them.
The harm is real. The accountability is not.
This gap has forced victims into an uncomfortable and extremely unfair position. Rather than being protected by systems designed to help them, they have to push for change themselves, such as sharing their stories in the media, building survivor networks, and lobbying for new legislation. Those people who have suffered the most harm become the activists for reform. This should not be the case, but this is exactly how things are.
As Flew (2021) argues, digital platforms are not simply technical tools but powerful intermediaries that operate across economic, political, and informational domains simultaneously. They are more than communication tools. They decide what content reaches audiences, who gets protected, and who gets exposed. Regulating them effectively requires governance frameworks that match their complexity and speed, something most current legal systems have yet to achieve.
The Global Problem No Single Government Can Solve
If governing deepfakes within a single country is difficult, governing them across borders is even more challenging.
Deepfake content has no boundaries. A video created in one country can be uploaded to a platform hosted in another, shared through messaging apps registered in a third country, and viewed by audiences around the world, all within hours. Even if a victim successfully gets content removed from one platform in one jurisdiction, it may already exist on dozens of other platforms that operate under entirely different legal frameworks.
As Flew (2021) describes, internet governance is caught between two competing visions: one that sees the internet as inherently global and beyond the reach of any single state, and another that insists national governments have the right to govern what happens within their own borders. The result is a governance gap that bad actors are very good at exploiting, hosting content just beyond the reach of the laws that would otherwise prohibit it.
This does not mean regulation is impossible. But it does mean that effective governance requires countries working together, not just individual countries passing individual laws. When the harm is borderless, the response has to be as well.
So Who Is Actually Responsible?
At this point, it is tempting to conclude that this is simply a problem of bad individuals misusing technology and that the solution is better enforcement against those individuals. But we cannot control what every person chooses to do with a technology once it exists. Governance does not work this way, and it is not a realistic goal here either.
What we can control is the environment in which those choices are made, and that is where governance becomes important.
Platforms are not neutral spaces. As Flew (2021) argues, digital platforms function as gatekeepers of the information environment, not merely passive hosts. When a platform prioritises engagement metrics without asking whether that engagement involves consent, it is making a choice. When a platform declines to invest in detection tools for non-consensual synthetic content, that is also a choice, and people are harmed as a result.
Just and Latzer (2016) take this further, arguing that algorithmic systems actively construct social reality by shaping what information is visible and what is filtered out. For deepfake victims, this means harmful content can be amplified and spread while remaining effectively invisible to those seeking accountability. Platform design is not simply a technical question. It is a political one.
This is why placing responsibility entirely on individual users fails. Advising people to think twice before posting photos is placing the burden on potential victims while leaving the structural conditions that enable harm completely unchanged. Good governance is about shaping those structural conditions, not policing individual behaviour after the fact.
Governing Deepfakes: What Must Change
The good news is that change is possible and has already taken place in some areas. Australia, for instance, has enacted specific legislation that classifies the creation and dissemination of deepfake pornographic content as a criminal offence, with a maximum penalty of three years in prison (New South Wales Government, 2025).
But legislation alone is not enough. Platforms need to develop scalable detection and removal tools, treating involuntary synthetic content as a major issue rather than an isolated case. Regulators need to think systemically and address the environments that allow harm to spread, rather than responding to individual incidents after the fact. Such issues are global challenges that require global collaborative solutions, so international cooperation must also keep pace with the actual needs.
The technology itself will not disappear, and that is fine. The goal is not to eliminate the technology. The goal is to build governance systems strong enough to shape how it is used and prevent it from causing harm to others.
Key Takeaway
Deepfakes are not simply a technical problem that can be solved with better technology. They are also a governance issue. The most important question is not how to stop AI from advancing — it will — but whether the institutions we rely on to protect people are willing and able to advance alongside it. That is a question about political will, not technical capacity.
We cannot control every person who picks up a powerful tool. But we can control the laws that define what is acceptable, the platforms that decide what spreads, and the international frameworks that determine whether accountability is possible at all.
The choices made now by platforms, lawmakers, and regulators will shape what the digital public sphere looks like for years to come. Technology will always move fast. The real question is whether governance is willing to keep up. Because right now, for too many people, it does not.
References
Crawford, K. (2021). Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press.
Department of Homeland Security. (2023). Increasing Threat of Deepfake Identities. In Department of Homeland Security. Department of Homeland Security. https://www.dhs.gov/sites/default/files/publications/increasing_threats_of_deepfake_identities_0.pdf
Flew, T. (2021). Regulating Platforms. Polity Press.
Just, N., & Latzer, M. (2016). Governance by algorithms: reality construction by algorithmic selection on the Internet. Media, Culture & Society, 39(2), 238–258.
Mackenzie, J., & Choi, L. (2024). South Korea: The deepfake crisis engulfing hundreds of schools. Bbc.com; BBC News. https://www.bbc.com/news/articles/cpdlpj9zn9go
Moore, A. (2025). “I don’t take no for an answer”: how a small group of women changed the law on deepfake porn. The Guardian; The Guardian. https://www.theguardian.com/society/ng-interactive/2025/dec/04/i-dont-take-no-for-an-answer-how-a-small-group-of-women-changed-the-law-on-deepfake-porn
NSW Government. (2025). NSW Government strengthens protections against deepfakes and image-based abuse. Communities and Justice; NSW Department of Communities and Justice. https://dcj.nsw.gov.au/news-and-media/media-releases/2025/nsw-government-strengthens-protections-against-deepfakes-and-ima.html
PASQUALE, F. (2015). The Black Box Society: The Secret Algorithms That Control Money and Information. In JSTOR. Harvard University Press. https://www.jstor.org/stable/j.ctt13x0hch.3
Westerlund, M. (2019). The Emergence of Deepfake Technology: A Review. Technology Innovation Management Review, 9(11), 39–52. https://doi.org/10.22215/timreview/1282
Be the first to comment