“Grok can put a bikini on everything.” On January 3, 2026, Elon Musk posted that and laughed. Because his AI had just dressed a toaster in swimwear, and apparently that’s what passes for innovation these days. The comments section, never one to miss a moment, ran with it: bikinis on cartoon robots, coding software, fictional characters. Peak chaos, peak 2026.
None of this was actually new. Since May 2025, concerns about Grok generating non-consensual images of real women had been circulating, and by January the situation had quietly tipped into something much harder to ignore. Reuters journalists who spent just ten minutes watching live public requests on January 2nd counted 102 attempts to digitally edit real people into bikini photos. That’s just what was visible on the surface. The Center for Countering Digital Hate estimated Grok had generated around 3 million sexualised images in total, roughly 23,000 of which appeared to involve children. Bloomberg’s reporting found it was producing approximately 6,659 nudified images every hour. That’s 84 times greater than the top five dedicated deepfake websites on the internet put together.
So, on the same day Musk was celebrating his toaster moment, that same tool had quietly become the world’s most prolific generator of non-consensual intimate imagery. The bikini toaster got laughs. The women got nothing except images of themselves they never agreed to exist. Same feature, same platform, same day.
The System Behind the Scandal
How does something like this even happen at this scale? It’s worth stepping back, because the answer isn’t really about one rogue AI or one bad decision. It’s about how these systems are built to work in the first place.
Just and Latzer (2017) argue that algorithms function like institutions, operating as “norms and rules that affect behavior on the supply and demand side, as a set of rules and routines that both limit activities and create new room for maneuver” (p. 244). What makes modern AI systems like Grok so distinct is that they operate through automation and, as Andrejevic (2019) points out, automation isn’t neutral. The system is deployed on a commercial platform optimized specifically to enhance user engagement and scalability, and its built-in features reflect these priorities. Grok generates countless images every hour, but there is no review process behind it at all. The reason is simple: in the era of runaway algorithms, who still considers manual approval a serious matter? The system just ran, and the harm scaled with it.
That’s where the black box problem makes everything so much harder to fix. Pasquale (2015) describes a black box as “a system whose workings are mysterious; we can observe its inputs and outputs, but we cannot tell how one becomes the other” (p. 3). Nobody outside xAI knows exactly how Grok decides what it will or won’t generate, which makes it nearly impossible for regulators to identify where things went wrong. As Flew (2021) notes, existing legal frameworks were built around personal data and privacy issues and are largely powerless when it comes to the kind of automated content generation that Grok represents. This law targets platforms that share content instead of those that generate it, and the significance of this distinction is far greater than most people realise.
How Grok Built for This
Aurora, xAI’s image generation model, has been part of Grok since September 2024. It was quietly integrated into X as one of its more distinctive features. Unlike many competing tools at the time, Aurora stood out because its content restrictions were comparatively limited. This allowed it to generate realistic human figures and complex visual scenes that other platforms would typically flag or refuse. By March 2025, xAI had further expanded Aurora’s reach by making it available through an API, allowing access beyond X itself. This design choice was not incidental. According to most accounts, it was once a major selling point of the product.
Concerns about the misuse of the tool to generate non-consensual images had already emerged in May 2025, but the situation escalated significantly in the final days of December. By January 2026, what began as isolated reports had developed into a documented crisis, with researchers, journalists, and advocacy groups identifying the same pattern from different perspectives. On January 1, Grok issued a public apology on X, acknowledging that it had generated sexualised images of minors and that this reflected a failure in its safeguards. Three weeks later, CBS News tested the feature and found that it remained functional, with image generation quietly placed behind a Premium subscription rather than being removed entirely. Aurora was not shut down, instead, it was repackaged.
When the Rules Don’t Apply
The Grok scandal didn’t expose a gap that nobody knew about. It exposed a gap that existing frameworks were never designed to close. Flew (2021) notes that current legal framework is built around personal data and privacy, and thus structurally appears inadequate when dealing with large-scale automated content generation. The problem goes beyond outdated wording. It comes down to a basic mismatch between how regulation sees platforms and what platforms like X actually do. Regulation treats them as distributors, but tools like Grok are more like manufacturers.
The global response to the scandal makes this problem hard to ignore. Governments across more than 12 jurisdictions rushed to act, but their responses ended up revealing the same underlying issue. Some countries, including Indonesia, Malaysia, and the Philippines, resorted to temporary bans. These measures signalled urgency, but most were reversed within weeks, demonstrating that straight forward strategies like blocking can shut a platform down but cannot govern it. In jurisdictions with stronger legal infrastructure, such as the EU, France, and Ireland, authorities launched investigations under frameworks like the Digital Services Act and GDPR. Although legal remedies already exist, law enforcement remains inherently reactive in nature. Investigations are only initiated after significant harm has already occurred, and legal proceedings often take years to resolve. The third category of governments—including the United Kingdom, Canada, and India—responded primarily through political rhetoric, such as public condemnation, calls for investigations, and, in some cases, threats of action, though these threats were ultimately not carried out. However, rhetoric is not a mechanism of governance. At the other end of the spectrum, Japan’s AI Promotion Act relies entirely on voluntary compliance with no enforceable penalties, making it perhaps the clearest example of what happens when soft governance meets a high-risk technology. Taken together, these four patterns point to the same conclusion: no existing model offers a way to intervene before harm occurs.
Part of what makes this so difficult to fix is what’s often called the “black box” problem. As Frank Pasquale (2015) points out, “interlocking technical and legal prohibitions prevent anyone outside such a company from understanding fundamental facts about it” (p. 8). In short, these systems are designed to be hard to see into, and that’s not accidental. More transparency could weaken a company’s competitive edge. The result is a major challenge for regulators. If they cannot see inside the system, they have no way of knowing where failures take place, or how to stop them in the future. Shifting toward more proactive regulation would depend on greater transparency, but that is exactly what companies are least willing to provide.
What makes the reform even harder is the commercial logic running underneath all of this. On X, engagement drives revenue, and sexualised, controversial content generates exactly the kind of clicks and time-on-platform which advertisers value. This is evident from the fact that xAI decided to restrict Grok’s image generation capabilities to paid Premium subscribers rather than simply removing it. The harm has been monetised, rather than being addressed. As long as platforms can profit from harmful content and retain legal protections that allow their systems to remain opaque, proactive governance will largely remain a pipe dream.
Who Actually Gets Hurt
While structural issues and regulatory limits help explain how the problem emerges, they do not fully account for who is most affected and why. This is where the role of algorithmic bias becomes important.
Noble (2018) argues that algorithmic systems are never neutral. They are built by people, and the assumptions, values, and biases those people hold are inevitably embedded into the technology at scale. What appears to be a technical design choice is often a social choice in disguise. In the case of Grok, the Aurora model was designed to generate realistic human figures with relatively few restrictions. When users directed it at real women and girls, the system responded quickly and at scale. The harm was not random. It followed a familiar pattern, disproportionately affecting those who are already more vulnerable to sexualisation and less likely to have access to legal or institutional protection.
Andrejevic (2019) adds another layer to this argument: automation does not simply execute instructions but also reflects the social and economic context in which it operates (p. 26). On a platform optimised for engagement, content that sexualises women tends to perform well. The system learns from this, amplifies it, and the cycle reinforces itself over time. This is not a malfunction, but the system operating as designed within a context where gender-based harm has long been undervalued and underregulated. As Just and Latzer (2017) observe, Algorithmic systems often amplify existing inequalities rather than eliminate them, yet it is difficult to determine who is responsible for the resulting outcomes. If the business logic underlying such incentives were to change, and if gender-sensitive safeguards were incorporated during the system design phase rather than as an after-the-fact remedy, this model would likely continue.
So, What Now?
What happened with Grok wasn’t a glitch, and it wasn’t a surprise. It was the expected result of a system that designed to reward engagement, operate behind closed doors, and move faster than any regulatory framework could follow. The apology came. The subscription arrived. Long before either of those things happened, the damage had already been done on a large scale to real people.
Fixing this requires more than patching individual bugs. Platforms that use generative tools like Aurora should have to tell the public how their models are trained, how content is filtered, and what risk assessments were done before the models are made public. This kind of transparency that Pasquale (2015) argues is systematically withheld because opacity is so commercially valuable. Responsibility also needs to shift. When a system’s design choices directly result in the generation of harmful content, the responsibility should lie with the platform, not with the individual users who happened to make the request. And gender-sensitive safeguards need to be built into these systems from the start, not refilled after women and girls have already been harmed.
For anyone reading this who uses X, or any platform built on generative AI, my word isn’t to stop using the internet. It’s to start asking different questions — not just what these systems produce, but why they’re built to produce it, and whose interests are being served when they do.
Reference
Andrejevic, M. (2019). Automated media. Routledge.
Bloomberg Business [@bloombergbusiness]. (2026, January 9). Grok is posting 84 times more sexually suggestive AI images per hour than the top five deepfake sites combined [Photograph]. Instagram. https://www.instagram.com/p/DTQN8m0Emjl/
CBS News. (2026, January 26). X, Grok AI still allow users to digitally undress people without consent, as EU announces investigation. CBS News. https://www.cbsnews.com/news/x-grok-ai-imagery-elon-musk-eu-uk-us-regulation/
Center for Countering Digital Hate. (2026, January 22). Grok floods X with sexualized images of women and children.https://counterhate.com/research/grok-floods-x-with-sexualized-images/
Chan, K. (2026, January 6). Elon Musk’s Grok chatbot draws global backlash for generating sexualized images of women and children without consent. Fortune.https://fortune.com/2026/01/06/elon-musks-grok-chatbot-deepfakes-nude-images-women-children/
D’Anastasio, C. (2026, January 7). Musk’s Grok AI generated thousands of undressed images per hour on X. Bloomberg. https://www.bloomberg.com/news/articles/2026-01-07/musk-s-grok-ai-generated-thousands-of-undressed-images-per-hour-on-x
Flew, T. (2021). Regulating platforms. Polity Press.
Just, N., & Latzer, M. (2017). Governance by algorithms: Reality construction by algorithmic selection on the internet. Media, Culture & Society, 39(2), 238–258. https://doi.org/10.1177/0163443716643157
Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.
Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Harvard University Press.
Vicens, A. J., & Satter, R. (2026, January 4). Grok says safeguard lapses led to images of minors in minimal clothing on X. Reuters. https://www.reuters.com/legal/litigation/grok-says-safeguard-lapses-led-images-minors-minimal-clothing-x-2026-01-02/
Wilson, J. (2026, January 8). Hundreds of nonconsensual AI images being created by Grok on X, data shows. The Guardian. https://www.theguardian.com/technology/2026/jan/08/grok-x-nonconsensual-images
xAI. (2024, December 9). Grok image generation release. https://x.ai/news/grok-image-generation-release
Be the first to comment