Inside the invisible machine that decides what you see, what you believe and what keeps you scrolling.
By Yu Han
A close-up of a smartphone displaying a social media feed with algorithm-curated content. (How to land on the TikTok “For You Page” Lookfamed)
It’s 9pm on a Tuesday. You spend 5 minutes on YouTube.
A hour later you are still watching and somewhere in the process the videos became more angry, and more radical and more outrageous. You never chose the wrong path. But something did.
The something is a recommendation algorithm: a software program designed not to tell you, not even to deepen your knowledge of the world, but to make you continue watching as long as possible.
And as it so happens, outrage, fear, and sensationalism are very effective in doing just that.
“Algorithms don’t care about truth or your mental health. They care about engagement metrics.”
In today’s world, billions of people’s ideas about reality are formed by very few invisible computer programs (algorithms), with your emotions being used as raw material for production, while your attention is being sold to advertisers. There’s no conspiracy here; the platforms themselves have been aware of this for quite some time.
What Is a Recommendation Algorithm, Really?
Before we get into why this matters, let’s break down what we’re actually talking about.
A recommendation algorithm is just a collection of mathematical rules that a platform uses to determine what content will be shown to you next. Everywhere you go, every video you watch, every post you like, every time you stop on an image, and every time you click on a link is a data point that continually updates an algorithmic model that is made up of all the things you have done and continues to shape who you are.
A researcher named Natascha Just and Michael Latzer describes this as, algorithmic governance,which is a type of reality creation in that automated systems are used for wide-scale decision-making regarding what information people have access to, how their opinions are seen, and who’s voice is amplified (Just & Latzer, 2016).
So algorithms do not simply represent reality, they create it.
Another researcher, Frank Pasquale, who is a legal scholar, refers to these types of systems as, black boxes, because they are complex machines whose detailed processes are unknown by the general public and to the extent possible, unknown by the majority of those who develop the systems (Pasquale, 2015). You know what they do to you but you do not know how or why they do it.
The Engagement Trap: Why Outrage Is Good for Business.
TikTok’s ‘For You Page’ is driven entirely by algorithmic recommendations. (What happened on TikTok around the Romanian elections? | Global Witness )
The main issue at hand is that the goal of recommendation algorithms is not to provide you with reliable information but instead to produce high levels of engagement, such as having users click on, view, and share a video, as well as react positively or negatively to it. There is a high level of engagement with content that can elicit strong negative emotional responses (e.g., feelings of anger, fear, moral indignation) from viewers.
From the perspective of the platforms, when a video evokes an angry response from a user, that user is much more likely to comment on the video, post a negative comment to their own timeline, and/or continue viewing similar types of content. The algorithm observes this behaviour and provides more of the type of content that made the user react.
Over time, as you continue to interact with this type of content, the way your feed looks will gradually start changing to become more extreme, more emotional, and increasingly inaccurate in terms of representing the diversity of beliefs that exist in the world.
According to media theorist Mark Andrejevic (2019), this phenomenon is referred to as “automated culture,” meaning a culture that is not determined by humans or through public discussion but rather is determined by automated systems that sort content based on user behaviour rather than its public value.
“Studies show toxic videos get 2.3% more interactions than neutral content. Partisan posts receive roughly double the engagement of balanced perspectives.”
When Facebook Knew and Did Nothing.
The seriousness of this issue can be seen in the Facebook Papers, where a whistleblower provided thousands of pages of documents showing that the Facebook algorithm promoted hate, division and emotional reactions to posts with hate and exaggeration because they received more engagement from users.
Researchers at Facebook found that the “angry” emoji reaction, which was introduced in 2016, was used five times as often on misinformation, emotional harmful posts as on factual content because the algorithm was able to identify it as a success. In 2019, the company’s analytics scientists verified that postings that elicited an angry reaction emoji were more likely to contain harmful content, false information, and poor news (Chakradhar, 2021) .
In addition, Facebook researchers recommended modifying the algorithm to reduce the effect of angry reactions on engagement but were told not to pursue those recommendations because doing so could reduce the number of people using Facebook for advertising revenue. What this demonstrates is that Facebook has the data, researchers raised concerns based on that data, and the company chose to continue with the engagement model; therefore, this is not an accident, but rather a decision made by design.
‘The platform knew. It had the data. It had researchers raising alarms. And it chose engagement over safety.’
The Problem Is Getting Worse, Not Better.
Since Haugen’s disclosures, we would like to believe that these platforms have improved; nevertheless, that does not appear to be the case.
For example, the algorithm used by TikTok – the platform that is now most popular with youth worldwide has been reviewed in a 2024 audit and researchers concluded that it actively directs users toward progressively more extreme content through what they refer to as a “radicalisation pipeline” (Shin and Jitkajornwanich, 2024).
The important point is that a large portion of this extreme content being consumed by TikTok users was not searching for that type of content on their own through the TikTok application; it was recommended by the algorithm They consumed this content because they were recommended content based upon their individual activity, history, or stored geographic location. Therefore, the algorithm did not just follow your desire to see content; it was also causing you to access content that you didn’t request to see.
Another study of over 390,000 TikTok videos during the US presidential election in 2024 showed that there was political bias in the type of videos recommended to users based also on their watching history and location, with certain parties being preferred over others (Ibrahim et al., 2025)
Ultimately, the tools have changed; however, the incentive structure among these platforms is the same across the board.
The Real-Life Cost
This may appear to be abstract but isn’t.
The amplification of outrage and extremism through algorithms causes harm to real people. It increases political polarization. It fractures communities. It destroys trust in the shared reality.
The impact on youth is especially troubling. An Amnesty International report from 2024 found that TikTok and Instagram’s recommendation systems were rapidly providing vulnerable youth with self-harm material.
Reset Australia found that misogynistic and extremist content including far-right influencer content was pushed to young men’s feeds within minutes of their very first interaction (Regehr et al 2025).
These are not edge cases; when platforms are utilized by billions of users, even a minor nudge in an algorithm towards harmful content will have major detrimental effects in the real-world.
Accountability and Solutions: Social Media, Society, and Self-Regulation
To begin, it is fair to say that there will always be an argument for social media platforms to self-regulate. If this is the case, social media sites are already making strides in reliability with content moderation, community standards, algorithmic research partnerships with outside organizations, and third-party fact-checkers; however, it would appear that the Facebook Papers have revealed limits to self-regulation. Social media platforms have repeatedly chosen to pursue profit over the benefit of the public when profit and public interest conflict.
Due to the limitations in self-regulation, the need for an external governance mechanism is now warranted. To date, two external governance mechanisms are in place and will continue to grow as we move forward. One currently in place is the EU’s Digital Services Act, which became fully bridged into legality as of 2024. One of the major components of the DSA is that the largest social media platforms must conduct independent audits of their algorithmic systems and evaluate their systemic risks to the public. To date, the DSA is the largest regulatory intervention within social media platforms to date and is a relative “prototype” for other governments to emulate.
Another example of an external governance mechanism is Human-Centric Design Principles for Building Reliable Systems. Currently, researchers are examining alternatives to tomake systems the most optimal for users and society as a whole. The fundamental approach to this work is to redesign the “incentive structure” from the ground up.
Finally, one question that needs to be addressed is transparency. Currently, there is little to no transparency into how these systems work. Requiring these platforms to conduct algorithmic audits of their systems to allow researchers, journalists, and regulators to evaluate the systems would be a great first step.”
There is certainly a global challenge to coordinating efforts on global coordination since the Internet is unbounded by national borders. The Recommendation by UNESCO on the Ethics of Artificial Intelligence proposes such standards for algorithm accountability internationally but these standards will remain possibilities without internationally binding enforcement mechanisms in place.
The Algorithm Is Not Neutral
When you were sucked into that rabbit hole Tuesday night, it was indeed intentional and planned out. It wasn’t by accident; it wasn’t something you did; and it wasn’t coincidental.
Your attention was captured by a system designed to grab your attention at all costs — even at the expense of your mental health, your relationships, and the healthy function of civil society.
The purpose of recommendation algorithms is to provide neutrality. But recommendation algorithms are anything but a means to neutrally provide content from source to destination. Recommendation algorithms are also a major influencer and manipulator on millions of individuals, creating a changing landscape of what people believe, are fearful of, and value.
As we’ve clearly learned from the Facebook Papers, the developers of these harmful systems know they are doing harm and continue to prioritize that engagement produces profit over protecting the individual or the civil society.
To be clear: algorithms control our lives. The question is: will we now start to take action to control them?
We can do this through increasing transparency, meaningful regulation, and holding algorithm developers and decision makers accountable for not only their economic but ethical (or human) obligations to the people.
There are 100’s of engineers who are making decisions that impact the information that gets delivered to billions and will continue to influence all of us into the future.
This is not only a technology issue; it is a democracy issue.
References：
Andrejevic, M. (2019). Automated Media. Routledge. https://doi.org/10.4324/9780429242595
Chakradhar, S. (2021, October 26). More internal documents show how Facebook’s algorithm prioritized anger and posts that triggered it. Nieman Lab. https://www.niemanlab.org/2021/10/more-internal-documents-show-how-facebooks-algorithm-prioritized-anger-and-posts-that-triggered-it/
Ibrahim, H., Jang, H. D., Aldahoul, N., Kaufman, A. R., Rahwan, T., & Zaki, Y. (2025). TikTok’s recommendations skewed towards Republican content during the 2024 U.S. presidential race. ArXiv (Cornell University). https://doi.org/10.48550/arxiv.2501.17831
Just, N., & Latzer, M. (2016). Governance by algorithms: reality construction by algorithmic selection on the Internet. Media, Culture & Society, 39(2), 238–258.
Pasquale, F. (2015). THE BLACK BOX SOCIETY The Secret Algorithms That Control Money and Information. https://raley.english.ucsb.edu/wp-content/Engl800/Pasquale-blackbox.pdf
Regehr, K., Shaughnessy, C., Zhao, M., Cambazoglu, I., Turner, A., & Shaughnessy, N. (2025). Normalizing toxicity: the role of recommender algorithms for young people’s mental health and social wellbeing. Frontiers in Psychology, 16. https://doi.org/10.3389/fpsyg.2025.1523649
Shin, D., & Jitkajornwanich, K. (2024). How Algorithms Promote Self-Radicalization: Audit of TikTok’s Algorithm Using a Reverse Engineering Method. Social Science Computer Review, 42(4), 1020–1040. https://doi.org/10.1177/08944393231225547
Walowsky, M. (2020, May 8). How to land on the TikTok “For You Page”. Lookfamed. https://lookfamed.de/en/news/so-landet-ihr-auf-der-tiktok-for-you-page/
Global Witness. (2024, December 17). What happened on TikTok around the annulled Romanian presidential election? An investigation and poll. https://globalwitness.org/en/campaigns/digital-threats/what-happened-on-tiktok-around-the-annulled-romanian-presidential-election-an-investigation-and-poll/
Be the first to comment