In the current age of technology, algorithms decide what individuals see, and the posts that go viral, and who gets hired. Algorithms have often been considered to be objective because they are free from human prejudice and, thus, able to make data-driven and rational decisions. However, what if the opposite is true?
Algorithmic decision-making has prompted the emergence of new and complex forms of discrimination, most of which are hidden within the algorithms (Wang et al.,2024)
Discrimination emerging from algorithms manifests itself in different forms, such as magnifying disparities in healthcare, criminal justice and hiring. It is worth noting that AI systems that have been trained on historical data tend to amplify past prejudices, such as racial profiling in predictive policing or gender bias in hiring tools.
The Illusion of Algorithmic Neutrality
Algorithms are consideredneutral tools. It is believed that unlike humans, AI has no feelings and as such, cannot discriminate. However, this assumption ignores the reality of the fact that these algorithms are built and trained by humans and as such, they work based on data fed to them.
According to Helen Nissenbaum (2018), in her concept of “contextual integrity”, it is impossible to separate technology from the social contexts that they operate. This saying implies that data is not neutral, but instead, it encompasses information influenced by power relations, cultural norms and historical inequalities. The fact that algorithms depend on data implies that they also inherit these factors.
Another key issue is the lack of transparency in algorithmic system, an issue referred to as the “black box” problem. It is worth noting that most algorithms operate in ways that even creators themselves do not understand and this makes it difficult to identify when biases occur.
These poor operation limits accountability because people who are affected by these systems cannot question the decisions made about them. This outcome means that the perception of neutrality continues to exist, even when the outcomes suggest otherwise.
How Do Algorithms Reinforce Inequality?
Most AI models are trained using historical data, which reflects past discriminatory practices. An example is the hiring algorithm, which learned to penalize female candidates majorly because mostly men were hired in the past. Similarly, facial recognition systems have also been criticized to perform poorly on dark-skinned people, especially women. The reason behind this is that their training datasets are dominated by lighter-skinned faces and this results in underrepresentation of these groups and their exclusion from services.
The case is the same for predictive policing tools, which are often trained on historical policing data. These tools end up predicting higher crime rates in neighbourhoods that were over-policed in the past.
Health algorithms have also been found to underestimate the needs of certain populations. These algorithms are trained to use health spending as the baseline for proxy of illness. However, historically, some groups spend less on healthcare due to access or financial barriers and the algorithms end up assuming that they are healthy. This assumption results in unequal treatment.
In addition to the above examples, algorithmic bias also arises through the selection of variables that are used in decision-making processes. Even when we eliminate sensitive attributes,such as gender or race, algorithms still depend on proxy variables like income level, education, and location, which are still linked to these characteristics.
Therefore, there exists an indirect form of discrimination that makes it harder to detect and regulate bias, while also resulting in unequal outcomes across different social groups.
Baised training often results in feedback loops and mental “moral cover”. Biased predictions create actions that reinforce the predictions, resulting in issues such as more surveillance in certain areas. Concerning moral cover, individuals end up following the decisions of AI even when they seem biased.
Inequality Among the Marginalized
The marginalized groups suffer more from the effects of biased algorithms. In their work, Marwick & Boyd (2019) insist that individuals ought to understand privacy and data practices from the perspective of marginalized communities. Their research argues that privacy is not an individual choice, and instead is often influenced by structural inequalities, such as, gender, race, status, and access to resources.
Marginalized people experience more severe effects because discrimination caused by algorithms can silence certain voices. Systems used in policing, for example, can reinforce existing disadvantages. Therefore, in this way, algorithms deepen social divides, instead of bridging them.
Recently, people who are famous online end up getting job opportunities even when they lack the basic skills. Algorithms focus on their data, so that their content is maximized and easily accessible. For example, people who go viral, have a higher chance of being attracted by brands for marketing,even when they have no skills, biased algorithms have resulted in such people getting jobs and being sought for their popularity, at the expense of skilled job seekers.
Additionally, people who go viral, have a higher chance of being attracted by brands for marketing, even when they have no skills, biased algorithms have resulted in such people getting jobs and being sought for their popularity, at the expense of skilled job seekers.
Additionally, marginalised communities mostly have no digital literacy or digital resources that are necessary to understand algorithmic decisions. As such, this lack creates an additional layer of inequality because they are the ones who are more affected by biased systems and at the same time lack resources to challenge these decisions. These people remain vulnerable to discrimination where they are discriminated both offline and online because they have no legal or institutional resources.
Privacy and Personal Data
Discrimination in algorithmic systems is also based on control over personal data. Research by Chen and Cheung (2018) focused on user behaviour on WeChat and discovered that privacy perceptions vary across user groups and contexts.
The findings reveal that users tend to trade privacy for convenience, a very unfair trade-off. As a result, some users get more surveillance and data exploitation compared to others. Individuals that are more exposed digitally end up creating more data, and they end up being targets of algorithms.
Unequal control over personal data occurs when organizations collect large data sets, using proxies and reinforce historical biases in the process.Inadequate control of data by individuals encourages algorithms to use proxy factors, such as race and zip codes to produce unfair outcomes, and make it almost impossible for the affected individuals to challenge decisions.
Social media algorithms create feedback loops, which amplify economic, social, and political biases that are present in the. data they are trained on. These algorithms use private data to help maximize profit. As much, they tend to focus on content that is sensational and emotional, in order to encourage social divisions and disadvantage marginalized groups.
The commodification of personal data is also a key issue. This issue occurs when user information is considered a valuable asset for companied. As such, power shifts from the users to large technoogical organizations, who assess and manipulate the data at large scale.
At the end, users have no control over the way their information is used. On the other hand, organizations just continue to shape the preferences, behaviors, and opportunities of users even if these interests do not align with those of usesrs.
For example, algorithms focus on influencers and overlook outsider groups, resulting in information inequality. Information inequality means that people with low income or marginalized groups of color have low to no access to job opportunities, information or social services.
What About Human Rights?
Technologies such as analytics by algorithms also interplay with human rights. They influence the right to privacy and expression. Algorithms end of interfering with private information by using it to suggest preferences and this is a public issue.
Human rights violation is evident when algorithmic data determines who will be listened to and who will be silenced. It also challenges the right to privacy when it collects and processes personal data without the consent of individuals. It raises the questions of equality and fairness by affecting certain groups.
As a result, there is a gap between democratic oversight and technological power.
The enforcement of these human rights is hard due to the global nature of the digital platforms. Nations across the world have varying legal frameworks as well as levels of data protection and this make it hard to come up with a standard for accountability.
Asc such, there is a gap whreby organizations operate with limited oversight and increase the risk of violating human rights. There is the end for global cooperation and regulation as algorithms continue to expand across borders.
Why Bias Persists
To understand the way that algorithms reinforce inequality, we must look beyong the technology and also assess the systems in which it operates.
A majority of digital pletforms today exists solely to attain economic incentives. These platform want to maximize the engagement and profit of users. Aa a result, algorithms aim at prioritizing content that keeps users always active, and often centers on emotional or sensational material.
The result of this is structural bias, and this does not mean that developers intend harm, but implies that the system rewards some outcomes over others.
It is also worth noting that datasets used to train algorithms are mostly unrepresentative and incomplete. Mostly, marginalised groups end up being misrepresented, underrepresented or oversurveilled in harmful ways that distort algorithm outputs.
A lack of diversity in tech teams can also affect the issue even further, when the individuals designing algorithms fail to reflect the diversity of users, then blind spots are inevitable.
Bias from algorithms also continue to persist because of the fast pace of development of technology, which interferes with the ability to regulate these systems effectively. Companies often prioritise development and innovation over ethical implications and this results in the use of systems that fail to promote fairness.
At the end, this imbalance resulted in continual inclusion of biased algorithms in everyday decision-making before the implementation of corrective measures.
Can Algorithms Be Fixed?
If algorithms are prepetuating inequality, the next thing is to know if they can be reformed or if the problem is deeper than we thought.
Policymakers and scholars have proposed different technical solution to address the issue including:
- Developing ethical AI design frameworks
- Use of more representative and diverse data when training algorithms
- Implementation of bias detection and correction tools
It is crucial to represent all voices when discussing issues of technological change and impact.
According to study by Marwick & Boyd(2019),marginalized voices are crucial when deisgning policies on ways of handling technology.
When deisgning algorithms, it is crucial to clearly state their purpose, the factors considered in the design, and the impact on people. In this way, it will be easy to clearly highlight any unethical issues, such as inequality, and solve it at the designing phase before it becomes an afterthought.
Justice is key in Algorithms
The main aspects to be considered when achieving justice in technologies like algorithms include:
- Eliminate bias: the systems should never be trained on historically biased data because this would mean they perpetuate the same in their analysis. They should be a representative of the whole population, and as such, incorporate the views of each person and diverse data.
- Transparency: the systems should be open and easy to understand so that people can assess the way that decisions are made. Besides that, organization must state what they intend to use with the data collected and if it is in line with laws of privacy and information handling.
- Voice: algorithms must not just focus on equal outcomes, but also make sure that each person had the chance to understand, debate as well as offer their imput in matters that affect them. Examples include the parol data.
- Human-Like：artificial intelligence should never replace human judgment, for example, in cases like sentencing that can influence the outcome of a person’s life forever. The systems should be used side by side with human judgment, and not as a replacement. In this way, wthical oversight will be achieved.
- Corporate role: Global organizations are instrumental in AI because they play crucial roles in the way algorithms function. These organizations have access to large amounts of data and should be in the frontline to ensure that the focus is on justice and not capital gains. Small firms often have to suffer from decisions made by large firms in relation to algorithms because they don’t have the required resources to influence these policies.
The aim is not to eliminate algorithms. The goal is to ensure they serve the public good and don’t reinforce existing hierarchies.
References
Barker, G. (2025, May 5). The illusion of neutrality: AI, bias, and the myth of objective search. LinkedIn.
Chen, Z. T., & Cheung, M. (2018). Privacy perception and protection on Chinese social media: A case study of WeChat. Ethics and information technology, 20(4), 279-289.
Ginès i Fabrellas, A. (2024, September 5). Algorithms are driving inequality, not eliminating it. Esade. https://dobetter.esade.edu/en/algorithms-equality
Marwick, A. & boyd, d. (2019) ‘Understanding Privacy at the Margins: Introduction’, International Journal of Communication, pp. 1157-1165. Wang, X., Wu, Y. C., Ji, X., & Fu, H. (2024). Algorithmic discrimination: examining its types and regulatory measures with emphasis on US legal practices. Frontiers in artificial intelligence, 7, 1320277. https://doi.org/10.3389/frai.2024.1320277
Be the first to comment