Many people believe that social media merely recommends content based on interests. However, many users have encountered this situation: some topics they have never actively searched for still keep appearing on recommendation pages.
The Guardian reported in 2024 that the algorithms used by social media platforms are rapidly amplifying extreme misogynistic content. These videos usually contain intense anger towards women and are more likely to be sent to young male users.
After users continuously come into contact with these videos, their understanding of gender relations will gradually change. This change will further influence their behavior in real life.
This phenomenon indicates that recommendation systems actively shape the information environment that users come into contact with. Platforms will incorporate harmful content into the daily browsing experience, and then continuously reinforce and push similar content.
This article argues that social media platforms, in an effort to increase engagement and generate commercial profits, continuously amplify existing gender biases through a black-box recommendation system. This process alters the information environment that users are exposed to and also shifts the platform’s responsibility and governance issues to a more prominent position.
How Recommendation Algorithms Work
We need to first understand the operational logic of the recommendation algorithm before we can comprehend why this content is constantly being pushed to users. In the information society, vast amounts of data form the basis for the algorithm’s operation, and people are increasingly relying on algorithms to understand information (Just & Latzer, 2017, p. 248).
This means that platforms do not simply push content; platform algorithms determine which posts appear on which users’ homepages. The core of the platform algorithm lies in predicting the user’s engagement level, and the platform will prioritize pushing posts that it believes are likely to generate interaction (Narayanan, 2023, p. 18).
This kind of push notification method gradually shapes the information that users see every day. The content that users come into contact with is continuously filtered and repeatedly reinforced.
Just and Latzer (2017, p. 245) pointed out that while humans are shaping algorithms, they are also being shaped by algorithms. This viewpoint indicates that algorithms will enter the human judgment process.
As more data is collected and user interaction increases, the algorithmic process will be further optimized. As Narayanan (2023, p. 24) pointed out, user behavior records constitute the “fuel” of the recommendation system. Discussion areas related to misogyny often have high discussion volume and high forwarding rates. These signals will be recognized by the system as valuable content.
In a sense, user engagement is an alternative indicator for the platform to achieve its business goals. Users with high engagement are more likely to continue using the platform and to drive advertising revenue. Therefore, more interactive content is more likely to be continuously recommended (Narayanan, 2023, p. 18).
The platform may not understand the meaning of content that discriminates against women, but the system will track the interaction results. As long as this kind of content attracts more views, comments, and shares, it will be considered popular and receive more traffic.
Once users begin to come into contact with this kind of content, the algorithm will push more similar content to the front, thus forming a self-reinforcing feedback loop.
How TikTok Pushed Users Toward More Extreme Content
The research team from University College London (2024) and the University of Kent created TikTok accounts for different “teenage male prototypes”. These accounts were labeled with terms such as “masculinity” or “loneliness”, and over 1,000 videos were viewed on the recommended page within seven days.
The study found that initially, the content pushed by the system was roughly in line with the interests set by these accounts. However, just five days later, the recommended content became significantly more extreme.
The proportion of videos with misogynistic tendencies has risen from 13% to 56% in a short period. The report indicates that these videos involve objectifying women, sexual harassment, and demeaning women.
What changed in five days?
- Recommended content became more extreme
- misogynistic videos rose from 13% to 56%
- harmful videos included objectification, sexual harassment, and demeaning women
More importantly, this change occurred during the continuous recommendation process. When users watch videos, they often follow the content provided by the platform all the way through.
Over time, the information environment in front of their eyes will undergo significant changes.
Research from University College London indicates that platform algorithms can create a snowball effect. The algorithms package the content into light, entertaining forms and gradually push it towards more intense directions.
This process is worthy of attention because young users are still forming their own ways of judgment. The algorithms will take advantage of this and continuously amplify certain content.
Why Users Often Do Not Notice the Change
Recommendation algorithms have been influencing the information that users see. However, many users are not aware of how the algorithms operate. This invisibility is itself a part of the platform’s power. Pasquale (2015, p. 3) uses the term “black box” to describe a type of system whose internal operation process is difficult for the outside world to understand.
In the context of social media, this means that users usually cannot understand how the platform decides which content is recommended.
This opacity is not accidental. As Pasquale (2015, p. 6) pointed out, even if these systems can be technically explained, there are still significant difficulties. The algorithms themselves contain numerous technical structures and also involve the platforms’ commercial interests and legal protection. Pasquale (2015, p. 8) further explains that these intertwined technical and legal arrangements make it difficult for ordinary users to access the basic facts of platform operation.
In other words, the users are confronted with an unknown information system. This deliberately closed decision-making process leaves users unable to understand why certain content is presented to them, and also prevents them from effectively questioning these recommended results.
This also means that the platform has, in an invisible manner, gained control over the information flow, while users lack the corresponding right to know and the ability to question these outcomes.
In this situation, users can easily interpret the content they see as “naturally occurring” rather than as filtered results. For instance, in a study by The Guardian, users were merely browsing the recommended content normally and were hardly aware that their information flow had gradually shifted from ordinary content to more extreme misogynistic videos within a short period of time.
The less users understand recommendation logic, the easier it is for platforms to shape what they see.
As people increasingly rely on platforms for information, the influence of these systems on users’ cognition continues to grow (Pasquale, 2015, p. 14).
Image source: Pexels
How Algorithms Amplify Gender Bias
The platform amplifies misogynistic content not only because the system is recommending popular content, but also because the recommendation system is deeply intertwined with the existing power structure. Crawford (2021, p. 8) analyzed artificial intelligence and illustrated this point.
She believes that such systems serve the dominant interests. These systems need to be trained with a large amount of data and rely on preset rules and reward mechanisms to make judgments.
From this perspective, the recommendation system seems to be an objective technical tool to ordinary people, but in reality, it has always been influenced by the dominant interests.
This influence will also enter the process of the platform organizing information. When the platform organizes information, it will consider how the content is classified, which content appears first, and which content is repeatedly pushed.
These arrangements will all affect how users understand the world around them. Crawford (2021, p. 20) pointed out that artificial intelligence is re-drawing and intervening in the world, which is a political activity dominated by a few companies that possess technology and capital.
This means that the platform is also implicitly shaping users’ understanding of the world.
This platform logic also leads to biased content receiving more exposure. Noble (2018, p. 19) pointed out that search and recommendation systems often present content with racial or gender discrimination as the top search results.
She believes that this sorting process reflects a corporate logic; the platform either deliberately ignores such biases or will continue to reinforce this kind of dissemination under the drive of profit.
More importantly, this bias is not an accidental phenomenon. Noble (2018, p. 21) further pointed out that algorithmic oppression is not a malfunction of the system, but a reasonable part of the platform’s operational logic.
This supports the idea that the platform’s operating mechanism leads to the dissemination of discriminatory content. The stereotypes about women are also continuously reinforced during this process. The repeated content gradually alters people’s perceptions.
The originally biased expressions, when exposed frequently, seem to become common (Noble, 2018, p. 29).
Returning to the case of The Guardian, TikTok’s recommendation system continuously amplifies misogynistic content because such content is inherently related to the existing gender bias in society and is more likely to generate clicks, comments, and shares. The platform enhances the exposure of such content through repeated recommendations.
After users frequently come into contact with this kind of expression, their understanding of gender relationships will be affected.
Why Platforms Should Be Responsible
When the platform actively amplifies this misogynistic content through algorithms, we should consider who is responsible for the negative impacts caused by the dissemination of such extreme content. Although social media companies often explain such results as caused by the automated operation of the technical system, all the actions of artificial intelligence are trained through intensive computational processing of large datasets (Crawford, 2021, p. 8).
Just and Latzer (2017, p. 251) pointed out that algorithmic selection is rarely used for social and political governance purposes, but is mainly employed for purely commercial goals. When platforms prioritize user engagement and advertising revenue as their primary objectives, controversial content is more likely to trigger interactions.
Such content will receive an advantage in the system’s recommendation scores, making it difficult for the platform’s recommendation mechanism to maintain absolute neutrality.
Therefore, this problem has changed from a mere technical issue to a governance issue involving transparency, accountability, and platform responsibility. In the current context, where algorithms are closely tied to the public information environment, platforms not only need to conduct stricter content reviews but also bear the social consequences of their recommendation systems.
Platform responsibility is not limited to deleting illegal content, but should also include examining the logic of the videos themselves, because it is the platform’s recommendation mechanism that really determines the influence of the videos.
Why This Matters Beyond the Screen
After platforms repeatedly spread content that discriminates against women, these expressions will gradually enter real life through the screen. Remarks that were originally extremely offensive will gradually be accepted by users after repeated exposure.
This impact is not limited to the screen. As The Guardian shows, over time, young users will gradually regard sexist content as a common expression, and even as part of youth culture.
Although this study is mainly focused on TikTok, this conclusion can also help us understand similar phenomena on other platforms.
In the long run, if platforms always reward content that touches people’s emotions the most, the public information environment will become more and more inclined toward conflict. Such an information environment will narrow the space for understanding and discussion, and will also affect the way users judge social issues.
conclusion
The real concern is that algorithms will gradually increase the visibility of discriminatory content. What users see is not a neutral information environment, but a result that has been filtered and sorted.
In this sense, the problem of social media algorithms goes far beyond technical efficiency. Once a few companies have the ability to screen and distribute content, they will shape which content is more likely to be seen and which views are more likely to be accepted.
Such problems have entered the scope of power and governance.
In the face of increasingly accurate recommendation systems, users need to remain vigilant about platform content and cultivate the ability to distinguish information.
Reference
Crawford, K. (2021). The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.
Just, N., & Latzer, M. (2017). Governance by algorithms: Reality construction by algorithmic selection on the internet. Media, Culture & Society, 39(2), 238–258.
https://doi.org/10.1177/0163443716643157
Narayanan, A. (2023). Understanding social media recommendation algorithms. Academic Commons.
https://doi.org/10.7916/khdk-m460
Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.
Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Harvard University Press.
http://www.jstor.org/stable/j.ctt13x0hch
University College London. (2024, February 5). Social media algorithms amplify misogynistic content to teens. UCL News.
https://www.ucl.ac.uk/news/2024/feb/social-media-algorithms-amplify-misogynistic-content-teens
Weale, S. (2024, February 6). Social media algorithms ‘amplifying misogynistic content’. The Guardian.
https://www.theguardian.com/media/2024/feb/06/social-media-algorithms-amplifying-misogynistic-content
Be the first to comment