Have you ever had such a feeling –
You just casually clicked on a video, and then for the next half an hour, your screen seemed to understand you: the content you like kept popping up one after another, with almost no mistakes, and everything was pushed at just the right time.
You might think that this is just technological progress, that “AI has become smarter.”
But what if we think about it from a different perspective?
In fact, it’s not you who are choosing the content, but rather the content that is choosing you?
In today’s digital world, we come into contact with a vast amount of information every day – news, short videos, music, shopping recommendations, and so on. Almost none of this is random. In fact, it is all filtered, sorted, and pushed to us by an invisible system. These systems are usually referred to as “algorithms”.
The problem is that we almost never truly understand how these algorithms work.
More importantly, we seldom realize that:
they are not only “recommending content”, but also quietly shaping the world we see.
Algorithms are not tools but power
When we use search engines, watch short videos or browse social media, it is easy to view algorithms as a “neutral tool” – they merely filter information based on our interests. But in fact, this understanding is overly simplistic.
As Frank Pasquale pointed out in “The Black Box Society“, we are entering a society dominated by opaque systems. In these systems,
we can observe its inputs and outputs, but we cannot tell how one becomes the other.
In other words, we can see what we click and what results we get, but we cannot understand the decision-making process in between.
What is more notable is that this opacity is not accidental but closely related to the power structure. Pasquale (2015) further pointed out:
To scrutinize others while avoiding scrutiny oneself is one of the most important forms of power.
On digital platforms, enterprises can constantly collect user data and analyze behavioral patterns, but the logic of their algorithms remains closed to the public.
This information asymmetry has far-reaching implications. For instance, credit scoring systems, search rankings, or recommendation mechanisms are all quietly influencing individual opportunities and the distribution of social resources. However, those being evaluated often have no idea about the assessment criteria and are even less able to question the results.
Therefore, algorithms are not merely technical tools for enhancing efficiency; they also represent a new form of governance mechanism. Through hidden rules, they sort and shape information, opportunities, and even reality itself.
When we fail to understand these systems, it is also very difficult for us to truly supervise them.
When cultural choices are left to algorithms
If algorithms have become a kind of invisible power, then one of the areas they most directly affect is the “culture” we come into contact with every day.
Before the advent of the Internet, the filtering of culture was mainly done by people: editors, journalists, critics, DJs, teachers… To a certain extent, they determined what was worth seeing, hearing and discussing. But today, this task is increasingly being taken over by automated systems.
As Andrejevic (2019) pointed out, when we are confronted with a deluge of information, we:
offloaded much of the social work of shaping our cultural world onto automated systems.
That is to say, the “cultural screening work” that originally belonged to human society is now being replaced by algorithms.
This transformation is not merely an issue of efficiency but a profound structural change.
In an environment of extreme information overload, it is impossible for us to browse through all the content by ourselves, so we have to rely on recommendation systems to “choose for us”. However, the problem lies in that the operational logic of these systems is not a neutral public service, but is built on commercial platforms. Their goal is often to maximize the time users spend, click-through rates and engagement.
This also explains why social media and content platforms are increasingly inclined to push emotionally charged, controversial, and even extreme content. As relevant research has pointed out, such automated recommendation systems tend to:
prioritize the distribution of polarizing, controversial, false, and extremist content.
But what is more alarming is the impact of this mechanism on our way of thinking.
Andrejevic (2019) reminds us that the issue lies not only in what we “see”, but also in the conditions under which we receive this information and how these conditions shape our judgment capabilities. In other words, as we become accustomed to relying on algorithms to filter the world, we are gradually giving up the process of active understanding and judgment.
Algorithms are not only recommending content, but also reshaping the way we understand the world.
Automation is not “neutral”
Often, when we discuss algorithmic issues, we tend to interpret “bias” as technical errors – such as incomplete data or insufficient model training. However, this understanding overlooks a deeper problem: automation itself comes with specific inclinations.
As Andrejevic (2019) pointed out, the “bias” here does not merely refer to discrimination or error, but rather an inherent system logic. In the contemporary information economy, the core objective of automated systems is to process vast amounts of data, enhance efficiency, and make decisions within an extremely short time frame. Therefore, they are naturally inclined towards speed, scale, and control.
Under this logic, the focus of technology is no longer on “understanding“, but on “prediction“.
Audrejevic (2019) clearly pointed out that this transformation implies:
the displacement of comprehension by correlation, of explanation by prediction
This means that the system does not care about why something happens; it only cares about which data patterns can help it predict future behavior more accurately.
This sounds highly efficient, but the cost is equally obvious.
When “correlation” replaces “understanding“, algorithms can classify, score and make decisions about people without explaining the reasons at all. For instance, a system might label you as a “high-risk user”, but it won’t tell you why, and even the developers themselves may not fully understand.
More importantly, this bias is not an accidental occurrence but is closely related to the current economic model. Automated systems are designed to “generate value from data”, which means that the more data there is, the faster it is processed, and the more accurate the predictions are, the greater the commercial value.
Therefore, what we see is not an “objective and neutral” technological system, but rather a system oriented towards efficiency and profit.
When algorithms prioritize prediction and efficiency, those social values that are difficult to quantify, such as fairness and human judgment, will inevitably be overlooked.
Is AI really “intelligent”?
To understand the limitations of AI, we can go back to a story that happened over a hundred years ago.
At the end of the 19th century, a horse named “Clever Hans” astonished the entire Europe. It seemed to be able to solve math problems, recognize dates, and even spell words. People believed that this horse truly “thought”. But later research revealed that Hans did not actually understand the questions. It merely picked up on the subtle reactions of humans when they were close to the correct answers, such as expressions, postures, or changes in breathing.
It is not “thinking”, but reading signals.
Kate Crawford (2021) used this story to remind us that AI systems can also fall into similar misunderstandings. As she pointed out, this case reveals:
how biases find their ways into systems and how people become entangled with the phenomena they study
Systems that seem intelligent often merely learn the patterns in data and the preferences in human behavior.
This also explains why AI is often overestimated.
In many discussions, AI is described as a kind of “intelligence” that is close to or even surpasses that of humans. However, Crawford (2021) raised doubts about this: The so-called “intelligence” is not independent but embedded within social, cultural and political structures. The data that AI systems rely on inherently carries biases and inequalities within these structures.
More importantly, AI is not a technology system that is completely divorced from reality. It relies on a vast amount of human labor, resource consumption, and institutional support to function, but is often presented as an “automatic” and “objective” black box.
Therefore, when we observe AI making seemingly “smart” decisions, perhaps we should instead ask:
What exactly did it understand?
Or is it just like Clever Hans, repeating our own biases?
If AI merely amplifies and replicates existing data patterns, it is not creating new cognition but reinforcing the existing social structure.
When algorithm “shape reality” – the case of TikTok
If what was discussed earlier was theory, then TikTok is the most tangible real-world manifestation of these issues.
You may have had this experience too: Just after opening it for a short while, the recommended content starts to become “more and more accurate”, even to the point of being a bit creepy. You would think – Wow, this platform really understands me!
But the question is, does it really just “understand you”?
How TikTok’s Algorithm Figures You Out | WSJ
If you only watch one video about TikTok’s algorithm, this investigation from The Wall Street Journal is enough.
They did something very simple but very terrifying:
Using dozens of “robot accounts”, keep watching videos non-stop to see what happens to the algorithm.
The result is that TikTok hardly requires you to say “what do I like”.
It only needs to observe one thing:
How long did you stay on a certain video?
Even if you just glanced for a few more seconds or hesitated for a moment, the system would interpret this as a signal and then bombard you with similar content in a frenzy.
Gradually, your homepage will become increasingly “monotonous” and more extreme.
Even research and investigations have found that this recommendation mechanism sometimes leads people step by step to more intense and emotional content, such as self-harm, extreme diets, conspiracy theories, and so on.
This is what we mentioned earlier：
- Algorithms don’t need to understand you
- It only needs to find out “what can stop you the most”
The problem is that what can make people stop and pay attention is often not the most genuine and rational content, but the most stimulating one.
Even regulatory authorities have begun to worry about this. The European Union has already launched an investigation into platforms like TikTok because their recommendation systems might constantly push users towards “more extreme and harmful” content.
So now, think back to one question:
Are the videos you come across really what the world is like in reality?
Or rather –
Is that just a world that has been constantly magnified, filtered, and even distorted by algorithms?
TikTok is not merely about predicting your interests; it is gradually training your attention and even changing what you will be interested in.
When algorithms are shaping the world, what choices do we still have?
If the information world in the past was about “us seeking out content”, now it’s more like “content is actively finding us”.
The problem is that this process is neither random nor neutral.
From search results and recommended videos to the news and viewpoints we come across every day, these seemingly “naturally appearing” contents have actually gone through layer upon layer of screening and sorting. Behind these sorts, there are business logics, data models, and a whole set of decision-making mechanisms that we cannot see.
This does not mean that technology is bad; it does indeed make information acquisition more efficient and personalized.
But the question is:
When we get used to letting algorithms make choices for us, are we also gradually giving up our ability to make judgments ourselves?
More importantly:
When these systems are quietly influencing what we see and believe, to whom should they be accountable?
Perhaps it is difficult for us to completely “break free from algorithms”, but at least we can start to realize:
- Recommendations are not neutral.
- Correlation does not equal truth.
- Sometimes, the content you like is trained.
In a world profoundly shaped by data and automated systems, what may truly matter is not to completely escape algorithms, but to consciously reclaim the initiative, for instance –
When you come across a piece of content, ask yourself one more question:
Why is it this?
When attracted, pause for an extra second:
Is this something I truly want to see,or am I being led to look at it?
When algorithms are shaping the world, at the very least, we should be aware that they are also shaping us.
Reference:
Andrejevic, M. (2019). Automated Media (1st ed.). Routledge. https://doi.org/10.4324/9780429242595
Crawford, K. (2021). The atlas of ai : Power, politics, and the planetary costs of artificial intelligence. Yale University Press.
PASQUALE, F. (2015). The Black Box Society: The Secret Algorithms That Control Money and Information. Harvard University Press. http://www.jstor.org/stable/j.ctt13x0hch
The Verge. (2024). TikTok’s addictive design is under investigation in the EU. https://www.theverge.com/policy/874746/tiktok-addictive-eu-regulators-infinite-scroll-notifications-autoplay
Wall Street Journal. (2021). How TikTok’s algorithm figures you out [Video]. YouTube. https://www.youtube.com/watch?v=nfczi2cI6Cs
Be the first to comment