Image source: bbc news
In October 2019, 25-year-old South Korean celebrity Sulli (Choi Jin-ri) unexpectedly passed away at home. The news instantly shattered the glossy veil that K-pop has, and sparked intense discussion about the problem of online harms everywhere around the world. Having been once a component of f(x), the past few years of Sulli in the spotlight were the years when the girl underwent a painful test of the digital violence. The misogynistic mystery of cyber bullying sucked her into her life several years ago simply because of going braless, simply because of making her relationship public or because she insisted on voicing her own side.
However, we must move beyond the moral condemnation of individual keyboard warriors. Regarding digital governance, the death of Sulli may be regarded as an isolated case of mental illnesses and a serious system-wide regulatory issue. The society is being compelled to reconsider the necessity to control platforms by a set of important social events, which Flew (2021) has endeavored to mark out. The Sulli case is a bright example of how hate speech can begin as a single act of enmity, but then turn into a social weapon, in the instance where platforms lack control systems that could effectively counter the hate-speech attempts directed at particular groups of people.
It is not only an online trolls story, but it is a story of the inadequacy in the governance of the platforms. In this paper, we will examine the algorithmic complicity that was exposed by Facebook former employee, Frances Haugen, in her 2021 testimony before the US Congress and why we so desperately need a regulatory framework not simply about legal compliance.
Structural Analysis of Gender Double Standards and Systematic Witch Hunting
Not a singular act, but instead a pattern of gender harassment that is deeply rooted in platform structures, Sulli became the target of the violence. The female idols in K-pop industry have to play a role of ideal relative compliance, any form of a deviation is recognized through the algorithms and pushed on the forefront of controversy.
This is depicted in an absurd case of gender double standards. In 2018, an attack on Red Velvet member Irene (Bae Joo-hyun) simply because she had read the feminist novel Kim Ji-young, Born 1982, her photos were even publicly burnt. However, once actor Gong Yoo revealed that he has been cast in the film version of the book, the reaction of the people was astounding and even applauded. This contrast affirms the findings of Sinpeng et al. (2021): hate speech is not just offensive words but a type of speech that could harm individuals and groups both in the short and long-term, but platforms are sorely lacking in the availability of definitions and enforcement against hate.
Algorithms multiply this circumstance of double standard ad infinitum. Algorithms, structures of governance and cultures of the users of platforms are all connected in ways that result in toxic technocultures, as Massanari (2017) examined, will systematically produce and reproduce harassment against women. It is not just on platforms where harassment is done but rather in doing nothing, they become tacit facilitators of gendered witch hunts. Furthermore, this is not just the case with platforms. Pabian et al. (2026) also discovered that younger social media users have developed a mental condition believing that hate speech on the internet is normal, and most bystanders prefer not to report or even act on such an event but instead pretend that it does not exist. This implies that not just algorithms and platforms that tolerated violence, but another component in this infrastructure of harm was thousands of silent bystanders in the case of Sulli.
The Logic of Engagement: When Gender Conflict Becomes Platform Profit
How come that the social media giant, with the advanced base of AI technology, could not prevent the systematic abuse of Sulli over a period of time? The root cause of the conflict between the core business logic and the user safety of the platform is the contradiction existing between these two aspects. Violence against Sulli was no longer a malice of sporadic violence, but turned into algorithms and turned into a marketable and exploitable emotional commodity by algorithms in the era of big data.
This is a failure in governance as the structural decision of profit first is the main cause. The year 2021 saw the testifying of former Facebook product manager Frances Haugen before the US Senate, where one of the secrets of the industry was revealed: inside the platform, research proclaimed that the algorithms were clearly promoting hatred, but as the models are known to boost engagement, management tended to turn a blind eye to it. Being a commercial motivated form of governance, this model has left a severe vacuum in the responsibility of platforms where the need to achieve next-generation volume of engagement invariably trumps the need to safeguard the users.
Image source: bbc news
The more significant crisis is that algorithms are organizing the current biases in the society. Technology is not neutral as Safiya Noble (2018) claimed in her book Algorithms of Oppression, in fact, more than anywhere else, algorithms contribute to the existent social injustice, introducing misogyny into information streams systematically. Gender-related issues, being controversial in nature arouse extremely high interaction rates. Once a platform notices that such conflict would spawn huge amounts of comments and prolonged dwell time, this auto-classification as a high-quality content triggers the algorithm, to increase its reach. According to this mechanism, the unconscious biases present in society are literally being fed and amplified by algorithms, which creates what Crawford (2021) calls a registry of power , in which AI systems are simply serving to satisfy the already dominant interests.
That is why this tragedy of Sulli never stopped the platform. In terms of platforms, female idols witch hunts are not only junk content but are fuel to the activity of platforms. Grasping the principles of the platforms, as Suzor (2019) cautioned, is not associated with transparency and due process, and thus, the rights of marginalized groups are more likely to be overlooked. Big data is no longer a problem-solving instrument, it has turned out to be a catalyst of the misogynistic society.
From Legal Sanctions to Systemic Intervention: Governance Dilemmas and Global Perspectives
After the tragedy of Sulli, the debate on how to contribute to the governance of the Internet has become polarized, and one of the strongest reactions to this question is strident legal interventions. In 2022, China implemented a show IP location policy with the aid of a digital panopticon to prevent mischief online. The trend is observed on platforms such as X previously Twitter as it becomes more identity transparent concerning identity transparency, as global governance changes with national sovereignty becoming an essential factor in digital space.
But what happened in South Korea proves that it requires identity transparency only. South Korea disclosed that its draconian approach tends to attack symptoms instead of causes in response to its real-name law in 2012, which it declared unconstitutional as fruitless and causing users to move to foreign sites. Real-name registration can reduce costs to abusers, but does not solve basic biases in algorithms.
Conversely, the model of eSafety Commissioners in Australia favours co-regulation and Safety by Design. The strategy would like platforms to act in ways that support intervening at the level of algorithms, and percentage-wise decrease the recommendation strength of toxic content targeting vulnerable populations, stopping the dissemination of harm on a large scale.
Image source: eSafety Commissioner
The move towards preventing rather than assuming responsibility after an event has happened is one such aspect that is consistent with the duty of care in the paper by Woods and Perrin (2021), Regulating Big Tech. According to them, the platform should be regulated whether its design and business format and not only particular materials. There should be a duty of care by statute to prevent harm and foresee on platforms. This method compares to the Health and Safety at Work Act of 1974 in the UK: the employers make sure there is safety in the workplace and so the operators of the digital environments are to make sure that there is no systemic damage online. The problem in the case of Sulli was not just unwanted malicious messages that were nonetheless not deleted, but that the organization did not fully maintain the design to avoid coordinated gendered harassment. The algorithm boosted worst content and the system of reporting failed to tell how rogue content propagated in the system. The duty of care model changes the situation, whereby the question of who uploaded this content becomes irrelevant, but how the platform is made? Does it contribute to this harm?
Woods and Perrin framework indicates a radically new approach to the logic of governance: governance should not just make individuals accountable, but transform the very structural role of the platform. Unless platforms alter their profit logic that revolves around engagement and unveil the black box of their algorithmic suggestions, then regardless of the ttransparency of IP locations, women will still be subject to witch hunts in the shadow of algorithms.
Reframing Digital Boundaries: Towards Algorithmic Transparency and Shared Responsibility
It is not just the fall of an idol when Sulli is leaving, it is a deafening warning. In the digital realm, where algorithms prevail, when the rule of uncontrolled commercial forces prevails over the survival of a person in her basic needs, every person might become the next victim.
This tragedy helps us to realize that current platforms governance models have a fatal lag. We should leave the days of voluntary compliance of platforms, like Flew (2021) underlined, and develop a more regulatory co-regulatory approach. The model of duty of care, suggested by Woods and Perrin (2021), offers an operation-blueprint, and the essence of it is the transformation of the safety responsibility on the user and content level to the level of system design of the platform. This implies cracking open the algorithmic black box: regulators ought to compel platforms to perform regular algorithm audits, reveal the manner in which their recommendation systems treat hate speech, and force action in the event a large scale gendered attack is discovered, as opposed to just taking user reports and acting after it happens.
Furthermore, Sinpeng et al. (2021) will remind us as well that digital governance must be culturally sensitive. In the context of the patterns of harassment as accusing some cultural scene, such as K-pop industry, these patterns are inclined to bear peculiarities of camouflage specific to the globalized social media environment. Platforms cannot simply insist on a single set of universal global norms to deal with all complex local struggles, the more complex, culture-sensitive, forms of review mechanisms must be strengthened by increased cooperation with the local civil society and research centers in order to identify the context-specific versions of gendered abuse.
In her last interview, Sulli only said that she wished to be treated like an individual, with the right to think and freedom to express her authentic self, not as a commodity meant to be consumed by the mass (Jung, 2023). To honour that quiet plea, neither legal provisions nor technical codes alone will suffice. The redistribution of digital power, keeping algorithms in the cages of governance, and restoring dignity to each person who breathes behind the screen needs to be re-evaluated. When the many women who are the most daring to speak out have to be sacrificed to the price of digital civilization, then it is this civilization we need to fully reflect on.
Image source: persona:sulli
Reference
Crawford, K. (2021). The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press.
Flew, T. (2021). Regulating platforms. John Wiley & Sons.
Massanari, A. (2015). #Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society, 19(3), 329–346. https://doi.org/10.1177/1461444815608807
Noble, S. U. (2018). Algorithms of oppression: How Search Engines Reinforce Racism. NYU Press.
Pabian, S., Van Moorsel, E., & Van Zon, J. (2026). ‘It’s a lost case’: a qualitative study investigating perceptions of young adult bystanders on online hate speech on social media. Behaviour and Information Technology, 1–15. https://doi.org/10.1080/0144929x.2026.2614046
Sinpeng, A., Martin, F., Gelber, K., Shields, K., Dept. Media and Communications, The University of Sydney, & The School of Political Science and International Studies, The University of Queensland. (2021). Facebook: Regulating hate speech in the Asia Pacific (By Facebook).
Suzor, N. P. (2019). Lawless: The Secret Rules That Govern Our Digital Lives. Cambridge University Press.
Woods, L., & Perrin, W. (2022). Obliging platforms to accept a duty of care. In Oxford University Press, Regulating big tech: Policy responses to digital dominance (pp. 93–109).
Be the first to comment