Are Algorithms Really Fair?
Nowadays, Artificial Intelligence (AI) has been widely advertised as an objective, bias-reducing, high-efficiency technique. From suggesting to us what news to view and which films to watch to filtering job hunters and predicting criminal risk, AI algorithms have been involved in many kinds of social decisions. Many technology companies and government agencies have described AI as a tool that can increase efficiency, optimise decision-making, and reduce human error. However, this kind of technical narrative, which appears positive, has obscured a more complex truth:
“Artificial intelligence as we know it depends entirely on a much wider set of political and social structures. And due to the capital required to build AI at scale, and the ways of seeing that it optimises AI systems are ultimately designed to serve existing dominant interests. In this sense, artificial intelligence is a registry of power.” (Crawford, 2021, p. 8).
I will discuss about AI system will copy the inequality from the society, because they are depending on the data with bias, simplified classification system as well as the power structure that is highly concentrate by a technology company Over explaining the AI algorithms’ operation mode, and to analysis a case study about hiring algorithms with bias, could look on how AI technique could produce social impact in the real life.
How about if those kinds of objectives are misleading?
In our daily lives, even some decisions have a greater influence, yet people rarely question the decisions made by algorithms. This is one of the reasons AI becomes much stronger: the lack of suspicion. Algorithms are running silently, often unknown to anyone, but they use the method in subtle, gradual ways to impact people’s opportunities and life course.
. “Algorithmic selection is essentially defined by the automated assignment of relevance to certain selected pieces of information.” (Just & Latzer, 2016, p. 239).
However, this opaque system lacks robust transparency and accountability mechanisms and puts all the power in one person’s hands.
In fact, AI’s design, training, and application are highly dependent on the data given by humans, and on the specific economy and political environment. In other words, AI is not a technique that operates in a vacuum, but rather a system embedded in the social structure. As mentioned by Just and Latzer (2016), algorithms have become a new kind of governance mechanism over the automation, sorting and ranking of information, affecting how people understand the world and how to act. When algorithms are trained on real-world data, they will retain the Prejudice, inequality, and power that already exist in real society. In addition, many more scholars are beginning to question the “Myth of AI’s objectives” and argue that AI may be unintentionally reinforcing social injustice.
As AI technology has been widely applied across fields such as public governance and commerce, its social influence has become more evident. In this article, I will explore why AI is not a neutral technology.
“Biases baked into computer systems and used across industries can silently threaten entire populations with unclear repercussions.” (Mateen, 2018, p. 287).
As Craford (2021) writes, AI technology systems are often controlled by smaller technology companies, leading to further centralisation of technical rights.
Artificial Intelligence, algorithms and Datafication
To understand why AI is not “neutral,” first understand how it operates. Modern AI mostly relies on machine learning, which involves overanalysing large amounts of data to uncover patterns, but does not rely solely on manually set rules.
An algorithm is the mechanism for sorting and ranking information. Moreover, Latzer (2016) point out that an algorithm “automatically assigning relevance to information” decides what information can be seen, how it can be understood, and how it can influence the structure in real life. In addition, algorithms are not only processing data, but also shaping social awareness.
These processes are closely related to “datafication”. People’s daily behaviour, such as searching, shopping, and social interaction, will be captured as data for the AI system training. However, data is not natural. As Mateen (2018) points out, many algorithms are built on the assumption that reality is simplified, but these assumptions often include prejudice and can be amplified within the system.
In addition, if past data contains inequalities, algorithms will inherit those inequalities. AI is not an objective judgment, but is a reproduction of past patterns and social structures. This means AI cannot only correct the inequality, but might also favour logical or data-driven decisions over biased ones, thereby normalising the inequality. As time passes, the discrimination phenomenon will become harder to identify because it is hidden within the technical system, not in the human behaviours that are easy to see.
Why is Artificial Intelligence not “Neutral”
Firstly, an AI system highly depends on data, but data itself has historical bias. As Mateen (2018) points out, a type of algorithm that has been called “mathematical weapons of destruction” has three characteristics: it is not transparent, cannot be appealed, and has a disproportionate effect on disadvantaged groups. Those systems will allow individuals’ problems to translate into structural inequality across a wide range of applications.
Secondly, algorithms are often operated as “black boxes”. Pasquale (2015) noted that, in financial and information systems, algorithms’ complexity and confidentiality make them difficult for the outside world to understand or to supervise. Users cannot get feedback on why their loan application was rejected or filtered out, or even know how to fix the errors. This does not mean transparency; rather, it means the lack of accountability mechanisms in algorithms. When a technical system cannot explain or suspect, it will become a new type of right.
Lastly, algorithms not only impact decisions but also shape reality. However, as Latzer (2016) points out, algorithms influence social awareness when selecting and ranking information, becoming a kind of “reality-constructing mechanism.” This mechanism usually increases inequality and reduces transparency when improving personalisation and commercialisation.
Case study analysis: the inequality in the hiring algorithms
(AI resume screening for your job board marketplace 2025)
Artificial intelligence has a typical case study in the hiring industry. More enterprises are using automated systems to sort job seekers, increasing efficiency and reducing human bias. However, Mateen (2018) points out that such systems often embed subjective judgment into algorithms and have broader impacts in large-scale applications. For example, Kyle Behm’s case shows that algorithms can exclude job seekers outside the labour market based on personality test results.
When Behm applied for a job, because he did not pass the algorithmic evaluation, he was rejected by multiple companies. This test simplifies more complex personality characteristics into fixed categories and helps decide whether to hire accordingly. What is more serious is that these systems are shared across companies, leading to “systematic exclusion”.
Mateen writes that the danger of this kind of algorithm lies in the fact that, at first, the bias embedded in the system might affect people across a wide range of applications. In addition, approximately 72% curriculum vitae (CVs) are not read by humans, but are filtered by algorithms. This means job opportunities will be more limited depending on whether individuals meet the machines’ requirements. This case explains that algorithms will not only fail to erase bias but might also institutionalise it through automation. This has raised an important ethical question about how individuals are evaluated in digital systems. Personality tests have simplified complex human characteristics into simple categories, which might put individuals whose experiences and preferences do not match those categories at a disadvantage.
On the other hand, this case shows that emphasis algorithms have deeper problems when interpreting human behaviour. Personality tests assume individuals can be classified as having fixed traits, but in fact, human characteristics are much more complex and influenced by context. What is included in Behm’s answer may reflect his personal experiences, but not his actual working ability. However, algorithms treated those answers as objective indicators, resulting in unfair outcomes. At the same time, although algorithms seem neutral and scientific, they distort individuals’ real experiences. What is more important is that, because multiple organisations use these systems, their impacts are not restricted to a single decision but accumulate over time, increasing the scarcity of long-term job opportunities.
Presents algorithms as not only a technical issue but also a social issue with substantive consequences, and this is why Behm’s case study is important when discussing algorithms. When automation systems are used repeatedly across companies, rejection is no longer a personal experience but part of a broader structural model.
Reconsidering the role of Artificial Intelligence in society
As Artificial Intelligence continues to increase its impact, humans must reconsider their governance approach. The algorithm systems now lack transparency and accountability mechanisms. “Failing clear understanding of the algorithms involved—and the right to challenge unfair ones—disclosure of underlying data will do little to secure reputational justice.” (Pasquale 2015, p.22.)
On the other hand, Latzer (2016) noted that algorithm governance is changing the traditional model, making technology itself a mechanism for rights. This means Artificial Intelligence governance is not only a technical problem, but also includes political and social issues.
In Addition, it needs to:
- Increase the algorithm’s transparency
- Build an appeal mechanism
- Strengthen supervision
More importantly, what AI reflects is the social structure; if the inequality cannot be solved in real life, algorithms will only copy those problems.
This also shows that technical means alone are not enough to solve the inequality problem. Even improving algorithm design and reducing bias are important steps. However, they could not erase the fundamental social factors that cause data bias. If the inequality persists in society in the long term, it will likely be reflected in the data used for training artificial Intelligence. In addition, the algorithmic bias problem needs to be addressed by both technical and societal means. If there is no broader structural change, creating “fair” Artificial Intelligence systems might have little effect.
Conclusion
Artificial intelligence is often described as a neutral, objective technique, but this view has ignored AI’s social attributes. Algorithms operate based on past data. However, that data often contains inequality (Mateen, 2018). At the same time, the non-transparency of algorithm systems makes them tools of rights that are hard to supervise. (Pasquale, 2015) The case of hiring algorithms shows that AI not only copies bias but might also amplify its impact across a wide range of applications. In addition, understanding AI must go beyond the technical level and be analysed from a rights and governance perspective.
However, this significant problem is not whether Artificial Intelligence is smart or not, but rather who controls it, who benefits from it, and who is excluded from it. A lack of effective supervision can not only reduce inequality but also increase social division in the digital age. As Artificial Intelligence continues to permeate more fields of our daily lives, solving those problems becomes increasingly urgent.
Looking ahead, Artificial Intelligence could become more popular in fields such as medicine, education, and governance, raising concerns about fairness and accountability mechanisms. As more decisions are entrusted to the automation systems, the possibility of large-scale inequality is increasing as well. As to this, it is important to have a strict review of those systems’ design, implementation and supervision methods. Lack of appropriate supervision: artificial intelligence can not only reduce inequality, but also increase the difficulty of finding people and challenge methods.
Reference List:
AI resume screening for your job board marketplace. Codica Blog RSS. (2025, March 30). https://www.codica.com/blog/how-to-implement-ai-based-resume-screening-in-your-job-board-marketplace/
Crawford, K. (2021). The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press.
Copeland, B. J. (2026, April 8). Artificial Intelligence (AI) | definition, examples, types, applications, companies, & facts | britannica. Britannica. https://www.britannica.com/technology/artificial-intelligence
Just, N., & Latzer, M. (2017). Governance by algorithms: reality construction by algorithmic selection on the Internet. Media, Culture & Society, 39(2), 238–258. https://doi.org/10.1177/0163443716643157
Kenton, W. (2025, August 21). Understanding black box models: Definition, finance use, and examples. Investopedia. https://www.investopedia.com/terms/b/blackbox.asp
Mateen, H. (2018). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy: Cathy O’Neil. Broadway Books, 2016. 268 Pages. Berkeley Journal of Employment and Labor Law, 39(1), 285–292.
Pasquale, F. (2016). DIGITAL REPUTATION IN AN ERA OF RUNAWAY DATA. In The Black Box Society (pp. 19–58). Harvard University Press. https://doi.org/10.4159/harvard.9780674736061.c3
Pasquale, F. (2016). FINANCE’S ALGORITHMS: THE EMPEROR’S NEW CODES. In The Black Box Society (pp. 101–139). Harvard University Press. https://doi.org/10.4159/harvard.9780674736061.c5
Be the first to comment