“In contemporary society, an increasing number of crucial decisions are hidden within ‘black boxes’, and the operation methods of these systems are invisible and difficult to question for ordinary people.” (Pasquale, 2015). This article argues that AI recruitment systems are not neutral tools, but rather a power mechanism that redistributes social opportunities through data and algorithms.
When you submit a resume and attend an online interview, perhaps in an instant, you have already been screened through thousands of people.
It sounds incredible, but that’s the truth. Because the people who screen you are likely not human beings, but an algorithm. When your expressions, speech rate, words used, even your pause times are recorded in the system and quickly evaluated, you have no time to understand the algorithm’s standards and you don’t know why you were rejected.
This sounds like something from the future, but in fact it has already happened.
- What is AI Recruitment? Why is it becoming more and more common?
In simple terms, AI recruitment is using algorithms to assist companies in screening candidates. For instance, it analyzes facial expressions, voice tones, and language expression logic in video interviews, while also combining resume data and historical recruitment records to evaluate candidates and predict who is more likely to become an “ideal employee”. The large amount of screening work that was originally done by human resource personnel has been transferred to data-driven systems for execution (AlgorithmWatch, 2019).
On the surface, the reason why AI recruitment has become increasingly common in recent years is quite simple. For enterprises, the traditional recruitment process is often time-consuming and labor-intensive, requiring human resource personnel to sift through a large number of resumes, arrange interviews, and make judgments. However, the AI recruitment system can process massive amounts of data in an extremely short time, thereby significantly improving recruitment efficiency and reducing labor costs. Moreover, the algorithm is not affected by emotions, biases, or fatigue, and can thus make more rational and consistent judgments. In the context of the continuous growth of global data, this “replacing manual judgment with data” approach is regarded as a more efficient solution (Mayer-Schönberger & Cukier, 2014).
However, behind this “efficiency improvement” lies the shift of the decision-making process from human judgment to machine models. Just and Latzer (2019) pointed out that algorithms not only process information but also determine which information is considered “important” or “relevant”, which means that the system is not only screening people but also defining what kind of people are considered “qualified” or “excellent”.
2. The “objectivity” of AI is actually an illusion
The reason why many people consider AI to be “fair” is that they believe machines do not have emotions, and therefore will not exhibit biases or discrimination like humans do. This understanding seems reasonable, but it overlooks a crucial issue: AI does not “think for itself”; it merely learns and predicts based on data. In other words, AI’s judgments are not independent but are based on the existing data of human society.
As Noble (2018) pointed out, algorithms tend to reinforce rather than eliminate existing social inequalities. Since the training data usually comes from past human behaviors and decisions, if these data inherently contain biases, such as gender discrimination or racial inequality, then the algorithm will unconsciously replicate and magnify these patterns during its learning process. In the recruitment scenario, this means that AI may be more inclined to select individuals similar to those who were “successful employees” in the past, and these criteria themselves may inherently carry historical biases. Moreover, Crawford (2021) further pointed out that artificial intelligence is not a neutral technological system but is embedded in broader social, economic, and political structures. The data relied upon by AI is not naturally present but is collected and constructed through various institutional and power relations. Therefore, when algorithms are used to evaluate individuals, they are actually perpetuating these structural biases rather than objectively reflecting reality.
Therefore, the “objectivity” of AI is more of an illusion. It conceals the complex data sources, design logic, and power relations behind the algorithm, leading people to mistakenly believe that the decisions are neutral and fair. In fact, this data-driven judgment method is likely to merely perpetuate past human biases in a more concealed and less detectable form.
3. Black box question: You were rejected, but you don’t know why
In many cases, the decision-making process of AI is not explainable. That is to say, the system will provide a result – such as rejecting your application – but will not tell you the reason. Pasquale (2015) referred to this phenomenon as “Black Box Society”, to describe those algorithmic systems that are invisible to the public and difficult to understand.
The reason why this is dangerous is that it changes the way we understand “decisions”. In the past, when someone was rejected, they could at least ask the recruiter for the reason, and even in some cases file an appeal. But in an algorithm-driven system, this communication channel is often cut off. You don’t know how the algorithm judges you, nor can you challenge this decision, and it’s even difficult to determine who is responsible for this decision.
More importantly, this opacity is not an accidental technical issue but rather a structural feature of the algorithmic system. Many AI models, especially those based on machine learning, have highly complex internal operation methods, and even the developers find it difficult to fully explain their decision-making logic. At the same time, these systems are often controlled by enterprises, and their specific mechanisms are regarded as trade secrets, which further exacerbates the opacity (Pasquale, 2015). When these systems lack transparency, they not only may perpetuate biases but also may widen inequality without anyone noticing.
4. Case Analysis: When AI Begins to “Discriminate” – Amazon’s Recruitment System
If AI recruitment is merely an abstract concept, then the case of Amazon makes this issue very concrete. This case has been widely reported in academic and media discussions about algorithmic bias. A few years ago, Amazon developed an AI recruitment system, hoping to use algorithms to automatically screen resumes and thereby improve recruitment efficiency. The logic of this system seemed very reasonable: by analyzing the resumes of past successful employees, identifying common features, and then using these features to screen new candidates.
However, problems soon emerged. Research revealed that this system systematically lowered the scores given to female candidates during the evaluation process. For instance, when words like “women’s” (such as “women’s chess club captain”) appeared in the resume, the candidate’s evaluation would be reduced. The reason is not complicated – the training data used by the system came from the recruitment records of the past ten years, and these data themselves originated from an industry environment dominated by men. In other words, the algorithm did not “learn fairness”, but rather learned to replicate past biases.
However, problems soon emerged. Research revealed that this system systematically lowered the scores given to female candidates during the evaluation process. For instance, when words like “women’s” (such as “women’s chess club captain”) appeared in the resume, the candidate’s evaluation would be reduced. The reason is not complicated – the training data used by the system came from the recruitment records of the past ten years, and these data themselves originated from an industry environment dominated by men. In other words, the algorithm did not “learn fairness”, but rather learned to replicate past biases.
Ultimately, Amazon had to abandon this project. This incident clearly demonstrates that AI will not automatically eliminate inequality in human society; instead, it may reinforce these issues in more covert ways. As Noble (2018) pointed out, algorithms tend to perpetuate and magnify existing social biases rather than reflecting reality neutrally.
Ultimately, Amazon had to abandon this project. This incident clearly demonstrates that AI will not automatically eliminate inequality in human society; instead, it may reinforce these issues in more covert ways. As Noble (2018) pointed out, algorithms tend to perpetuate and magnify existing social biases rather than reflecting reality neutrally.
5. The Power Structure Behind AI
Crawford (2021) pointed out that artificial intelligence is not merely a technology; rather, it is more like a complex system composed of multiple elements – including the technology itself, the industrial structure that supports its operation, and the underlying power relations. In other words, AI is not just “doing things”; it is also redefining who holds the decision-making power. Many people tend to view AI as a “tool”, like search engines or calculators, thinking that it merely helps us complete tasks faster. However, if we look a little deeper, we will find that things are not so simple.
The operation of AI systems is inseparable from a large amount of data, and this data often comes from the daily behaviors of ordinary users, such as clicks, searches, and content uploads. Secondly, there are computing resources. Large-scale algorithm training requires huge computing power support, and these resources are usually concentrated in a few large technology companies. Finally, there is human labor – from data annotation to system maintenance, there is actually a large amount of unseen manual work behind AI.
6.The Culture of Automation: We Are Being “Predicted” and “Managed”
If AI recruitment is merely a specific application, then the greater change actually occurs at the societal level as a whole.
Andrejevic (2019) proposed that we are entering an “automated culture”. In this environment, systems no longer merely respond passively to our needs, but start to actively predict our behaviors, guide our choices, and even optimize our decisions. In other words, many of our actions are being “calculated” and “planned” in advance.
In the recruitment scenario, this change is particularly evident. The AI system does not truly “understand” a person; it doesn’t know your potential, your personality or your growth space. What it does is to convert you into a set of data and then match it with existing models. That is to say, it is not recognizing you as a person, but rather judging whether you fit a certain established “pattern”.
7.How Algorithms “Shape Reality”
The problem brought about by this pattern matching is that it not only affects individuals, but also has a deeper impact on how we understand “reality”.
Just and Latzer (2019) pointed out that algorithms are not only filtering information, but also “constructing reality”. That is to say, when algorithms constantly select certain information and ignore other information, they are actually determining what is important and what should be seen.
In recruitment, this mechanism is particularly direct. AI systems will judge who is “qualified” and who is “excellent” based on training data. But these standards are not objectively existing; they often come from past recruitment logic, industry preferences, and internal value judgments of the company.
Therefore, when AI makes decisions, it is not only screening candidates, but also constantly reinforcing a certain definition of “ideal employees”. And once these definitions are solidified, they will become increasingly difficult to change.
8. Why is this important for ordinary people?
Perhaps some may think that this is just an internal technical issue within enterprises. However, in reality, it has a direct impact on every ordinary person.Because what the AI recruitment system does is not merely about improving efficiency. It is determining who has the opportunity to get a job, who can enter a certain industry or who will be excluded from opportunities.
In other words, this is not just a technical issue but a social issue.
When more and more critical decisions are handed over to algorithms, our opportunities, resources, and even future paths may be redistributed by these systems. And this redistribution method is often not transparent and is not easy to detect.
9. AI Governance: What Can We Do?
Since AI has begun to play a role in such important fields, a key question is: How should we govern it?
Firstly, fairness. In the design and use of AI systems, efforts must be made to avoid discrimination based on gender, race or background. This means not only focusing on whether the results are fair, but also checking for any biases in the data and models themselves.
Secondly, explainability. Users should have the right to know why they were rejected and how the algorithm made its judgment. Only when the decision-making process can be understood and examined can the system truly be trusted.
Thirdly, human involvement. AI can assist in decision-making, but it should not completely replace human judgment. Especially in scenarios involving important life choices, humans still need to retain the final judgment rights.
Since A ultimately refers to policies and regulations. For instance, the AI bill proposed by the European Union adopts a risk classification approach, imposing stricter regulatory requirements on high-risk systems. Meanwhile, international organizations are also attempting to establish a broader governance framework.
10. UNESCO: AI must comply with ethics
At the global level, the United Nations Educational, Scientific and Cultural Organization (UNESCO) has proposed a set of ethical guidelines for artificial intelligence, attempting to establish fundamental principles for the development of AI. “AI should serve humanity, rather than being the one that controls humanity.”
Conclusion
The AI recruitment system appears to be a technological advancement, but it is also a new form of power.
It not only screens candidates, but also determines who has the opportunity to enter a certain industry, access resources, and even participate in social operations. In this process, algorithms are no longer just tools; they become an implicit “decision-maker”, constantly making choices in the unseen places.
Reference List
Andrejevic, M. (2019). Automated media. Routledge.
AlgorithmWatch. (2019). Automating society: Taking stock of automated decision-making in the EU. https://algorithmwatch.org/en/wp-content/uploads/2019/02/Automating_Society_Report_2019.pdf
Crawford, K. (2021). The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.
Flew, T. (2021). Regulating platforms. Polity.
Just, N., & Latzer, M. (2019). Governance by algorithms: Reality construction by algorithmic selection on the Internet. Media, Culture & Society, 39(2), 238–258.
Mayer-Schönberger, V., & Cukier, K. (2014). Big data: A revolution that will transform how we live, work, and think. John Murray.
Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.
Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Harvard University Press.
Thouvenin, F., Früh, A. (2019). Towards principled regulation of automated decision-making (ADM). University of Zurich.
UNESCO. (2021). Recommendation on the ethics of artificial intelligence. https://www.unesco.org/en/artificial-intelligence/recommendation-ethics
Be the first to comment