Deepfakes, Desire, and Data. How is AI Reshaping Pornography and Why is Regulation Failing?

Deepfakes, desire, and data in an unregulated digital economy

Artificial intelligence is no longer confined to everyday tools like ChatGPT, which now attracts over 190 million daily users, demonstrating its rapid integration into daily life; it is increasingly reshaping one of the most intimate and controversial spaces online: pornography.

This expansion is not neutral, but reflects a broader shift in how data, identity, and power are organised within digital systems.

CHAT GPT processes 1+ billion queries daily, with 190.6 million daily users 

– SQ Magazine, August 2025

With minimal effort, users can now generate explicit images, manipulate bodies, and insert real people into sexual content using deepfake technology. What once required production teams can now be done instantly, cheaply, and anonymously.

As this technology expands across digital platforms, the question is no longer whether it will be used, but who is regulating it, and why current systems are failing to keep up.

---

AI Porn Explained: From Content to Data

AI-generated pornography refers to sexual content that is either fully synthetic or digitally manipulated using artificial intelligence.

Deepfakes, in particular, allow users to superimpose a person’s face or likeness onto another body, often without their knowledge or consent through machine learning techniques such as autoencoders, which are trained on large datasets of human faces to identify patterns in facial features, expressions, and structure.

A 2023 study found deepfake pornography makes up 98% of all deepfake videos online, and 99% of the individuals targeted in deepfake pornography are women.

By learning how to compress and reconstruct these images, they can generate highly realistic synthetic faces, enabling the replication, manipulation, and reassembly of human likeness at scale.

“Deepfakes threaten to collapse the distinction between reality and fabrication.”

– Danielle Citron (2019)

At the centre of this is datafication, the process of transforming human bodies, identities, and behaviours into data that can be stored, analysed, and reused (Flew, 2021).

In practice, this means that images of real people become raw material for AI systems, capable of being endlessly reproduced, altered, and circulated. This transforms human identity into a manipulable resource, raising questions not only about consent, but about who controls and profits from digital representations of the body.

This marks a significant shift. AI pornography is not simply a new genre of content, it represents a structural transformation in how sexual media is produced.

As Kate Crawford argues, artificial intelligence should be understood not as a neutral tool, but as an infrastructure of power shaped by data extraction and social systems (Crawford, 2021).

This suggests that AI pornography is not simply a technological development, but part of a broader system in which data extraction and platform economies shape how bodies are represented and circulated.

In the context of AI pornography, this means that sexual content is not simply created, but constructed through systems that organise data, reproduce bias, and shape how bodies and identities are represented.

From representation to production: how AI reshapes desire

If traditional pornography is about representation, AI pornography is about production.

Users are no longer passive consumers. They can:

• generate highly specific scenarios
• customise bodies and identities
• simulate real individuals

This transformation shifts pornography into a system where desire is not only represented but actively produced through technology.

This is where algorithmic governance becomes central. Algorithmic governance refers to the use of algorithms to organise, prioritise, and regulate content and behaviour within digital systems. In this case, algorithms determine what can be generated, what is visible, and what is restricted.

As Tarleton Gillespie notes, platforms are not neutral intermediaries but active participants in shaping user behaviour and content visibility (Gillespie, 2018). Applied here, this means that AI pornography platforms are not simply distributing sexual content by hosting it, but they actively structure the boundaries of desire by determining what can be generated, seen, and circulated.

Research supports this shift, describing AI pornography platforms as systems that determine which fantasies can be materialised and circulated (Lapointe et al., 2026) . Desire itself becomes filtered, shaped, and scaled through technological systems.

In this sense, platforms are not neutral spaces of expression, but systems that prioritise certain forms of content based on engagement, profitability, and technical design.

As shown in Figure 1, although 72.4% of platforms prohibit content involving minors, a much smaller proportion restricts other forms of harmful content, with only 24.5% banning non-consensual or sexual assault scenarios.

This suggests that while some risks are clearly recognised, others remain inconsistently regulated, reflecting a fragmented system of governance (Lapointe et al., 2026). This suggests that governance does not consistently prioritise harm prevention, but selectively regulates content based on legal risk and platform interests. In this sense, platforms act as arbiters of which desires can be materialised.

Crucially, these dynamics extend beyond the platform itself.

---

Case study: when AI becomes criminal

These dynamics are not abstract, they are already producing real-world harm. In 2026, a case in North Carolina demonstrated the consequences of AI-generated sexual content.

David Tatum, a child psychiatrist, was sentenced to 40 years in prison after using AI tools to generate explicit images of minors. He digitally altered ordinary images, such as school photographs, into sexualised content by modifying images into synthetic child pornography. (U.S. Department of Justice, 2026).

This falls under CSAM (Child Sexual Abuse Material), defined as any visual content, real or synthetic, that depicts or simulates the sexual exploitation of a minor. The law treats AI-generated CSAM as equivalent to real abuse.

This case highlights an important contradiction.

On one hand, existing laws are capable of prosecuting extreme cases of AI misuse. On the other, they operate only after harm has occurred. This frames harm as an individual act rather than a systemic issue, obscuring the role of platforms and technologies in enabling abuse at scale. Regulation, in this sense, is reactive rather than preventative.

What regulation currently exists in this case?

Understanding why regulation is failing requires looking at the different layers of governance currently in place, particularly within the context of the Tatum case study:

Regulation Without Coordination: The Failure of Digital Governance

Defined by the Bureau’s Internet Crime Complaint Center (IC3), run by the FBI, existing legal frameworks within North Carolina prohibit:

• the production of CSAM
• the possession and distribution of exploitative material

This case shows how digital governance operates as a layered system, rather than a single, coherent framework. In the Tatum case, regulation functions primarily through criminal law, where U.S. legislation prohibits the production, possession, and distribution of child sexual abuse material (CSAM), including AI-generated content. This is enforced through agencies such as the FBI’s Internet Crime Complaint Center (IC3), alongside initiatives like Project Safe Childhood, launched by the U.S Department of Justice in 2006, which coordinate action across federal, state, and local authorities.

However, this model is fundamentally reactive. It intervenes only after harm has already occurred, focusing on punishing individuals rather than addressing the systems that enable harm in the first place. This is where a key gap in digital governance becomes visible. While the state assumes responsibility for enforcement, platform governance, the rules and technical systems platforms use to shape content and user behaviour (Gillespie, 2018), remains inconsistent and largely self-regulated across AI pornography platforms.

This creates a structural imbalance. Platforms provide the infrastructure that enables the generation and circulation of harmful content, yet are not held to the same level of accountability as the individuals who use them. Responsibility is therefore shifted onto users, while platforms continue to operate with limited oversight.

This imbalance reflects a broader logic of platform capitalism, where companies retain control over infrastructure and data, while externalising responsibility for harm onto users.

At a global level, organisations such as the United Nations have recognised AI-generated sexual abuse as an emerging issue. Yet this recognition is not matched by enforceable, coordinated regulation. As a result, these different layers of governance, legal, platform, and global, operate in isolation rather than as an integrated system.

Taken together, this reveals a deeper failure of digital governance: not a lack of regulation, but a lack of coordination and accountability. Harm is addressed after it occurs, while the systems that produce it remain largely intact.

Platform governance: fragmented control

AI pornography platforms largely govern themselves. Research shows:

• moderation practices vary significantly
• content restrictions are inconsistent
• responsibility is often shifted onto users (Lapointe et al., 2026) 

This creates inconsistent rules across platforms where harmful content may be restricted in one space but permitted in another. Platforms effectively act as private regulators of sexual content, without consistent oversight with audiences acting as intermediaries.

Global governance: recognition without enforcement

At a global level, organisations such as the United Nations have identified AI-generated sexual abuse as a growing crisis.

Deepfake pornography:

• disproportionately targets women
• spreads rapidly
• is difficult to remove once online

According to UN Women (2026), fewer than half of countries have laws addressing online abuse, and even fewer specifically cover AI-generated deepfakes .

This creates a significant gap between recognition and enforcement. This highlights the limits of global governance, where issues are acknowledged politically but lack enforceable mechanisms across jurisdictions.

The problem is widely acknowledged, but governance mechanisms remain limited and inconsistent.

---

Why current regulation is failing

The failure of regulation stems from a fundamental mismatch between technological development and legal frameworks (Flew, 2021) reflecting a broader tension in which technological systems often outpace legal regulation (Lessig, 2006). This reflects a deeper structural issue, where regulatory systems are designed to manage outcomes, while technological systems are designed to maximise scale, speed, and engagement.

Legal frameworks lag behind technology

Many laws addressing image-based abuse were developed before generative AI. As a result:

• deepfakes fall into legal grey areas
• consent is difficult to define
• enforcement becomes inconsistent

Platforms lack accountability

Platforms often rely on user reporting systems, placing the burden of regulation on individuals rather than institutions. Content removal is slow, and accountability mechanisms remain weak.

Digital identity is unprotected

AI enables the replication and manipulation of faces, bodies, and identities. However, there are limited legal protections governing digital likeness. This leaves individuals vulnerable to exploitation without clear legal recourse, revealing a significant gap in legal frameworks, where individuals lack enforceable rights over their digital likeness despite its increasing economic and technological value.

Governance is reactive rather than preventative

The Tatum case demonstrates that current systems intervene only after harm has occurred. There are few safeguards preventing harmful content from being generated in the first place.

---

What needs to change?

Addressing AI-generated pornography requires a shift from reactive enforcement to proactive governance.

According to UN Women (2026), key changes include:

Conclusion: regulating power, not just content

AI pornography is often framed as a content problem, but this misses the point.

What is emerging is not just a new form of media, but a new system of power, one that operates through datafication, algorithmic governance, and platform control.

As Kate Crawford argues, AI is not neutral. It reflects and reinforces the structures that produce it (Crawford, 2021). In this context, pornography becomes part of a broader data-driven economy in which bodies are transformed into data and desire is shaped through algorithmic systems.

The real issue, then, is not simply what AI pornography shows, but what it does.

It reshapes how identity is constructed, how consent is understood, and how harm is distributed.

The technology is already here. The harm is already happening.

What is at stake is not only the regulation of content, but the regulation of systems that define how identity, consent, and desire are constructed in digital environments.

What remains uncertain is whether governance can evolve quickly enough to respond, not just to the content, but to the systems of power that produce it.

References

Australian Broadcasting Corporation. (n.d.). The alarming rise of AI apps creating explicit images of real people. https://www.abc.net.au/listen/programs/latenightlive/jo-bartosch-pornocracy-grok-ai-porn/106446862

Canadian Centre for Child Protection. (2017). Survivors’ survey: Executive summary. https://protectchildren.ca/pdfs/C3P_SurvivorsSurveyExecutiveSummary2017_en.pdf

Crawford, K. (2021). Atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.

Citron, D. K., & Chesney, R. (2019). Deepfakes and the new disinformation war: The coming age of post-truth geopolitics. Foreign Affairs, 98(1), 147–155.

Digital Regulation Cooperation Forum. (2024). The future of synthetic media.

ECPAT International. (2018). Towards a global indicator on unidentified victims in child sexual exploitation material: Summary report. http://www.ecpat.org/wp-content/uploads/2018/03/TOWARDS-A-GLOBAL-INDICATOR-ON-UNIDENTIFIED-VICTIMS-IN-CHILD-SEXUAL-EXPLOITATION-MATERIAL-Summary-Report.pdf

Flew, T. (2021). Regulating platforms. Polity.

Gillespie, T. (2018). Custodians of the internet: Platforms, content moderation, and the hidden decisions that shape social media. Yale University Press.

Krietzberg, I. (n.d.). Revenge porn. The Streets.

Lapointe, V. A., Dubé, S., Petit, A., Kessai, T., Rukhlyadyev, S., Gravel, V., & Lafortune, D. (2026). The governance of AI-generated pornography platforms: A content analysis. New Media & Society. Advance online publication. https://doi.org/10.1177/14614448261421873

Ofcom. (2024). Deepfake defences: Mitigating the harms of deceptive deepfakes.

United Nations Women. (2026). Technology-facilitated violence against women and girls (report).

U.S. Department of Justice. (2006). Project Safe Childhood. https://www.projectsafechildhood.gov

U.S. Department of Justice. (2026). Child psychiatrist sentenced for using AI to create child sexual abuse material. https://www.justice.govVargas, I. (2024, August 26). AI and the future of sex. MIT Technology Review. https://www.technologyreview.com/2024/08/26/1096526/ai-sex-relationships-porn/