The hiring process has never been fair.
But at least when a human rejected you, there was someone to blame.
Imagine you have just spent hours meticulously tailoring your resume and crafting the perfect cover letter to submit to your dream job. Within hours you have received an email, a polite rejection telling you that you’ve been unsuccessful in your application.
What you probably don’t realise and what most companies don’t say is that a human had not once looked at your application. Instead, your cover letter and resume were scanned, graded and it was determined that you weren’t a good fit for the role.
Artificial Intelligence has transformed the way companies recruit, the World Economic Forum reports that approximately 88% of companies already use some form of AI for initial candidate screening. With roles sometimes receiving hundreds, if not thousands of applications at a time, employers have turned to AI and automation to assist with managing the application process to sort, screen, and rank candidates. While on paper this is reasonable: faster screening and less administrative burden, in practice this can have significant consequences for real job seekers.
What are AI hiring tools
AI hiring tools can come in many forms. Some tools scan applications for specific keywords and will rank applicants to ensure hiring compliance and efficiency. Others such as iMocha are used to analyse video interviews to assess a candidate’s tone of voice, facial expressions, and word choice to help determine suitability for a role. They are now being extended to the background check portion of job applications, creating behavioural reports on candidates by researching a candidate’s digital footprint.
Hiring used to be personal, AI tools are the opposite of this. They are reductive and each tool transforms the human experience into a set of data points, these points are then fed into a system which is allowed to make consequential decisions about a person’s future.
Van Dijck (2014) calls this process “datafication”, where human behaviour, and identity are transformed into data which can be processed, ranked and analysed. In the context of hiring, your years of experience, personality, and communication style become variables which are measured by a system. But can these variables accurately measure a good employee?
Companies understandably have adopted AI hiring tools to assist with hiring at scale. With the large volume of applications a company will receive for a job application, human screening can at times be genuinely impractical. However, what we also forget is that these tools were also sold to help remove human bias from the equation of hiring and increase the chances of finding the perfect candidate. Ironically, like a hydra, by eliminating one form of bias, they introduced another that can arguably be far harder to detect.
The Illusion of Objectivity
One thing to understand about AI systems in their current form is that they rely on data and algorithms to generate an output. These systems train on historical data sets and past records (hiring decisions) to make predictions on who would be the best candidate within a predefined parameter set up by the hiring company. Kate Crawford (2021) argues that AI systems are neither autonomous nor rational and depend entirely on the political and social structures they were designed with and ultimately only serve the existing dominant interests.
Safiya Umoja Noble (2018) builds on this by arguing that algorithmic tools encode the assumptions and priorities of the people and institutions that make them. The objectivity of algorithms is defended behind the argument that the decisions are made based on mathematical and scientific solutions, but if algorithms are developed by human beings, and the data they use to make decisions is created by human beings, can the output ever truly be neutral?
So who actually bears the cost of this? Sony et al. (2025) identify several groups who are harmed by this bias, including women, neurodiverse individuals, non-binary people, and older workers. This list is rather broad and the shocking part is that it also includes a significant portion of the workforce of today. In many cases the bias is both different and connected: resumes were downgraded when they contained indications of female identity, neurodiverse candidates were penalised for non standard speech patterns, older candidates were filtered out from opportunities. But let’s look at how this played out in practice.
Amazon’s “sexist” AI tool
In 2014 Amazon had set out to develop a tool that could be used to screen multiple resumes at once and produce the top rated candidates who would be hired for the role. By 2015, they realised the tool wasn’t rating candidates in a gender neutral way and by 2018 the tool was scrapped because it had taught itself to discriminate against women.
So what happened?
The models used to assist in hiring decisions were trained on the historic data of resumes submitted to the company from the past 10 years. With the tech industry being predominantly male dominated, a majority of the resumes received were understandably male, based on this data the system taught itself that male candidates were preferable and would penalise and downgrade any resume that included the word “women’s”. Amazon attempted to fix the existing model but were unable to guarantee that it wouldn’t devise other ways of sorting candidates in a discriminatory manner and ultimately decided to close the project. The fact that Amazon determined they couldn’t fix the system shows everything about how our existing inequalities can embed themselves into systems sold to us as objective.
HireVue AI video interviews
HireVue is a recruiting company used by some of the world’s largest brands such as Unilever, Vodafone and Goldman Sachs. It offered video interviews that are analysed using a proprietary algorithmic model to assess candidates’ communication skills, personality traits and overall job aptitude. It would collect this data by analysing facial features, movement and speech patterns as well as accent to assess candidates. The implicit benchmark for these assessments were standard English and neurotypical behaviour, what about people who don’t fall into this category?
For candidates with disabilities affecting facial muscle movement or non-English speakers, the effects can be profound. In one instance, a complainant in the United States of America who was deaf, spoke in non-standard English with a “deaf accent” and was automatically rejected by an automated HireVue assessment and received feedback to improve “active listening”.
The repeated scrutiny into HireVue and its facial analysis resulted in a complaint to the FTC by the Electronic Privacy Information Center and the facial analysis screening was ultimately discontinued in March 2020.
iTutorGroup – When bias is deliberate
We’ve so far looked at cases where bias in AI systems is a product of historical trends or due to the complexities and variance of human existence, but what if the bias was intentional in the design of the system?
The iTutorGroup is a company which provides English language tutoring to students in China. It was discovered that the company had designed their hiring software to automatically reject applications from women aged 55+ and men aged 60+ and had rejected over 200 applicants in the US due to their age. The US Equal Employment Opportunity Commission filed a lawsuit against the iTutorGroup and agreed to settle for $365,000.
What this case study reinforces is the idea that the bias in AI hiring tools is a result of the dominant interests, in this case the hiring company.
Takeaways
What these case studies highlight is that bias in AI hiring tools is not a single, fixable problem. It can emerge quietly from historical data, it can be baked into the assumptions of what a “good candidate” looks, sounds and communicates like and sometimes it can be entirely deliberate.
Ultimately, these systems are simply a reflection of society and the economy and we are willingly handing our livelihoods over to these systems.
Why is this so hard to fix?
As we have seen, the issue itself is complex and layered in ways we don’t realise, the issue itself isn’t technical, really it’s structural and the systems in place make it difficult to even know where to start.
The first problem is transparency. Frank Pasquale (2015, p. 3) accurately describes these systems as “Black Box”. We can observe the input and output but have no way of understanding how we get there. In the case of AI hiring tools, a candidate knows the documentation they submitted and has received a rejection, but has no meaningful way of knowing why they were rejected.
This issue is further exacerbated by the fact that the algorithmic methods are considered trade secrets (Pasquale, 2015, p. 4). They are actively and legally protected from scrutiny so even if discrimination occurs, the mechanism that caused it is protected from investigation. A company can cause harm to job seekers and face no obligation to explain how.
The second problem is that our legal and governance frameworks were not designed to handle the challenges raised by algorithms and AI use. Terry Flew (2021) notes three specific challenges. The first is whether a person affected by an algorithmic decision even has a legal right to challenge it. The second is bias and transparency, which is difficult to guarantee given the enormous imbalance of power between the institutions who build these systems and the people subject to their decisions. The third is accountability, when a machine makes a decision who is responsible for it?
We can see how the accountability gap is compounded as anti-discriminatory law was designed under the assumption that the decision makers are human. As Pasquale (2015, p.8) notes, decisions that were once made by humans are now being automated. Existing laws such as the Racial Discrimination Act 1975 are typically built for human actors, and emphasise that a “person” has made the discrimination. Now that the decision maker has become an algorithm, it becomes difficult to place responsibility and existing law struggles to respond.
All this results in a situation where we see the harm, we can see what causes it, but accountability is out of reach. Companies are protected by trade secrets, governance frameworks are designed for a different era and the people most affected are left with few options.
Where does this leave us?
Considering all of this, it’s not fair to say that AI has no place in hiring. There are clear benefits as managing thousands of applications manually is genuinely impractical and human bias is a well documented concern. However, replacing one form of bias with another and making it invisible is not progress.
The real questions we should be asking are who decides what a good hire looks like, whose historical patterns should these systems learn from, and how do we deal with people who fall outside these patterns.
The next time you apply for a job and don’t hear back or receive a rejection immediately, it’s worth considering: did a person actually see it? And if the result was made via an algorithm what would you do?
While we may not have these answers just yet, we should be asking them more than we currently are.
References
AI job recruitment tools could “enable discrimination” against marginalised groups, research finds – ABC News. (n.d.). Retrieved April 12, 2026, from https://www.abc.net.au/news/2025-05-08/ai-job-recruitment-tools-could-enable-discrimination-research/105258820
AI Video Interviews: The Future of Smart and Scalable Hiring. (n.d.). Retrieved April 11, 2026, from https://www.imocha.io/blog/ai-video-interviews
Applicant Tracking System ATS Software Australia. (n.d.). Retrieved April 11, 2026, from https://employmenthero.com/products/applicant-tracking-system/
Barnes, P. (2019, November 11). Artificial Intelligence Poses New Threat to Equal Employment Opportunity. https://www.forbes.com/sites/patriciagbarnes/2019/11/10/artificial-intelligence-poses-new-threat-to-equal-employment-opportunity/
Crawford, K. (2021). The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press. https://ebookcentral.proquest.com/lib/usyd/detail.action?pq-origsite=primo&docID=6478659
Flew, T. (2021). Regulating Platforms. Polity.
HireVue Discontinues Facial Analysis Screening. (n.d.). Retrieved April 12, 2026, from https://www.shrm.org/topics-tools/news/talent-acquisition/hirevue-discontinues-facial-analysis-screening
Hiring with AI doesn’t have to be so inhumane. Here’s how | World Economic Forum. (n.d.). Retrieved April 11, 2026, from https://www.weforum.org/stories/2025/03/ai-hiring-human-touch-recruitment/
How Fama’s Online Screening Solution Detects Misconduct at Work. (n.d.). Retrieved April 11, 2026, from https://fama.io/post/how-famas-online-screening-solution-detects-misconduct-at-work
In re HireVue – EPIC – Electronic Privacy Information Center. (n.d.). Retrieved April 12, 2026, from https://epic.org/documents/in-re-hirevue/
Insight – Amazon scraps secret AI recruiting tool that showed bias against women | Reuters. (n.d.). Retrieved April 12, 2026, from https://www.reuters.com/article/world/insight-amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK0AG/
iTutorGroup to Pay $365,000 to Settle EEOC Discriminatory Hiring Suit | U.S. Equal Employment Opportunity Commission. (n.d.). Retrieved April 12, 2026, from https://www.eeoc.gov/newsroom/itutorgroup-pay-365000-settle-eeoc-discriminatory-hiring-suit
Noble, S. U. (2018). Algorithms of Oppression: How Search Engines Reinforce Racism (1st ed.). New York University Press. https://ebookcentral.proquest.com/lib/usyd/detail.action?pq-origsite=primo&docID=4834260
Pasquale, Frank. (2015). The black box society : the secret algorithms that control money and information. Harvard University Press.
Racial Discrimination Act 1975 – Federal Register of Legislation. (n.d.). Retrieved April 12, 2026, from https://www.legislation.gov.au/C2004A00274/asmade/text
Sony, M. M. A. A. M., Amin, M. Bin, Ashraf, A., Islam, K. M. A., Debnath, N. C., & Debnath, G. C. (2025). Bias in AI-driven HRM systems: Investigating discrimination risks embedded in AI recruitment tools and HR analytics. Social Sciences & Humanities Open, 12, 102082. https://doi.org/10.1016/J.SSAHO.2025.102082
Tutoring firm settles US agency’s first bias lawsuit involving AI software | Reuters. (n.d.). Retrieved April 12, 2026, from https://www.reuters.com/legal/tutoring-firm-settles-us-agencys-first-bias-lawsuit-involving-ai-software-2023-08-10/
van Dijck, J. (2014). Datafication, dataism and dataveillance: Big Data between scientific paradigm and ideology. Surveillance & Society, 12(2), 197–208. https://doi.org/10.24908/ss.v12i2.4776
Be the first to comment