Every TikTok video, job application that never receives feedback, or automatically denied benefits claim shares a common element: an algorithm that makes invisible decisions. This algorithm is not simply part of a sci-fi movie’s world. It is an artificial intelligence whose automated decisions affect how we work, how information is shared, and how social services are provided.
But is it really as neutral, fair, and reliable as people assume?
Automated decision-making is becoming more common, especially in high-stake situations. There is an important, unspoken truth in our world: Technology is not neutral. Algorithms are a governance system, not an arbitrary tool. When algorithms are not combined with human decision-making, justice, and accountability, they become tools of oppression. Algorithms are especially dangerous in this age of automation. It is critical to the social structure of justice to understand how governance and algorithms interact to protect people’s rights.
What Is Algorithmic Governance?
The general public may not be aware, but we do live in a time of algorithmic governance. There are AI systems used for hiring which filter job applicants. There are systems for driving jobs and social media platforms that present curated, personalized feeds. In fact, even governments have begun implementing algorithmic systems for social welfare determined for eligible candidates and calculating debt assignment. Algorithms are being used in high-stakes decision-making processes, replacing, and in many cases, augmenting, human judgment. To what degree do these systems govern lives, dignity, and access to resources with so little public scrutiny? These systems do have enormous public impacts but operate quietly, and rely on little transparency. People interact with algorithms each day and yet struggle to explain how these systems operate.
Data-driven decisions and algorithmic systems are being used to govern the majority of people’s lives. Most people are unaware algorithmic thinking has become the low-hanging fruit for structuring decision-making processes. People interact with these systems to an extent that they have become accustomed to the constrained possibilities these systems provide. People may be unfairly deemed ineligible for opportunity, support, inclusive financial products, and employment, and they are unaware of the rationale. The decisions governing their lives are highly opaque, algorithmically determined, and the governing systems and their impacts are not widely visible to the people.
In The Atlas of AI, Kate Crawford identifies much of the misunderstanding of AI technology by dissecting the systems behind AI and revealing how the technology is not ‘artificially intelligent.’ AI lacks the ability to autonomously understand, as there is no consensus regarding the origin of the data. Large-scale data is evacuated at great expense, both monetarily and human resource-wise to construct systems of resource-consuming labyrinths. Determined at the hands of the wealthy and governmental powers, the so-called intelligence is only a derivative analysis of regressive-informed data. While designers and architects desperately attempt to convey the systems as apolitical, there are always strong ties to the prevalent hegemony. Crawford argues AI is a force of extraction like an industry as an alternative for nature. Instead of minerals, AI operates on data and human labor. Illusions of neutrality are accompanied by technology that reflects the status of power and control at the initiation of the systems. Technology has the inherent ability to control and police a society so when the technology directs the control, power is lost. In a cycle, as control is lost society delegates to the technology more power, specifically algorithms. This shift is not only a social issue, but a political and technical issue as well. With the implementation of the algorithm, governance is no longer in the hands of a person but of the algorithm itself. Questions abound regarding the implementation of a technology when the algorithm is in control. After all, governance is based on control of the technology, accountability is a blurred line, and society is left with a control that affects their lives immensely.
The Myth of Neutrality: Why Algorithms Are Not Fair
An inherent misunderstanding regarding algorithms is their supposed objectivity. Algorithms are assumed to be more neutral than people because they lack sentiment and malintent. This myth is particularly dangerous in this age of AI. Algorithms aren’t created to be biased. However, they are trained on large datasets that reflect society’s biases. The real world is rife with inequality, and the data that algorithms learn from is full of discrimination and prejudice in the form of structural or cultural biases. Algorithms do not create injustice, they are engineered to reproduce it.
Historical data reflects gender bias, racial bias, and systemic inequities. Algorithms learn from this data and also replicate or amplify it. This technical bias under the smoke screen of objectivity is insidious. Studies have shown discriminatory outputs from search engines, hiring algorithms biased towards men, and facial recognition systems that are more erroneous for people with darker skin. Algorithms do not create bias, they standardize it. This makes inequality more insidious, more scalable, and more difficult to confront. Algorithms are created to reproduce injustice and are insidious because they have the false legitimacy of coming from a machine.
The Robodebt Scandal — A Failure of Algorithmic Governance
Examining the Robodebt Scandal in Australia highlights the defaulting failures of algorithmic governance. Between the years of 2015 and 2020 the Australian Federal Government irrationally automated the detection and recovery of welfare overpayment debts. This program was shut down after ruling it to be unconstitutional. Robodebt is considered to be one of the most remarkable failures in Australia’s public service history. With no legal backing, and no ethical or humane considerations, Robodebt was advertised as an efficient use of technology to remove fraudulent welfare debts.
Robodebt’s main operational dysfunction was its algorithm that averaged incomes of people that the government debt collectors claimed were underemployed. This resulted in hundreds of thousands of people being legally and financially abused by the government. Many Public Servants and legal scholars warned the moratorium abuse. But the system continued to run mainly without any human oversight. Officials made a decision that cost over care, and the need to make quick decisions overcame the need to care responsibility.
More than 450,000 Australians, especially from the most vulnerable segments, including low-income earners, the disabled, single parents, and the elderly, have been affected by the consequences. The debt register included false debt claims that ruined people’s finances, caused extreme psychological pain, relationship breakdowns, and, in some circumstances, collapsed. The pressure that people were under from the insolvent and self-destructing state system drove people to suicide. The widespread suffering and trauma inflicted by this system reveal a devastating failure to account for human cost. This harm is not accidental—it is the direct result of large‑scale technological and governance failure.
Forced repayments of illegally collected debts were granted by the government in 2020, but the widespread trauma remained unaddressed. In September 2025, an additional anti-trauma payout of A$475 million was awarded to four victims of a settled class action lawsuit by the Australian federal government. A subsequent royal commission found that the scheme was illegal from the outset, and that senior government officials, including former Prime Minister Scott Morrison, knew about the unlawful scheme and allowed it to continue. The Robodebt scandal clearly demonstrates why automated systems must be accompanied by human review.
In the discussion of Robodebt, the obvious concern is the technical blunder, however, the true concern is the failing of governance. Exposing automated systems to decision-making of numerous, sensitive issues without the rudimentary elements of governance caused the Australian Government to accept collateral damage of trivialized scale. While an algorithm may be able to utilize data, they cannot ascertain the meaning of the data. They can apply rules, but they cannot judge the overall context. When decisions are delegated to machines that cannot comprehend the stakes, what is sacrificed is accountability—not efficiency. It is not possible for algorithmically vulnerable individuals to negotiate or advocate for themselves. Even if social structures are rigid and unequal, justice cannot be automated, and human dignity cannot be calculated.
Algorithms, Platforms, and the Threat to Public Discourse
The same challenges in governance show up in digital platforms. Social media and content platforms use algorithmic recommendations to determine what content users consume. This shapes users’ behavior, perception and influences public discourse. When algorithmically constructed, invisibility becomes the mechanism for constructing reality. The personalization of feeds, shapes and constricts users in the pursuit of predefined objectives, the information lack is violent, shattering differing conceptions that socialize public life.
In the digital age, platforms have expanded the roles they play in the information ecosystem, but the opacity with which they operate is concerning. The popularity of generative AI is automating the production and distribution of information. This hinders efforts to uphold the integrity of the information ecosystem. Users’ abilities to detect AI-generated content and other information that is purposefully misleading are becoming more and more refined, but the platforms are refining and increasing the amount of content that is manipulative. This content may be emotionally manipulative and/or created with deepfake technologies. It ultimately leaves the user vulnerable and informs a more unhealthy public sphere.
Global Regulation and the Future of Responsible AI
To counter the impact of generative AI and other new technologies, governments have begun to regulate the entire AI ecosystem. In the EU, the AI Act and UNESCO’s AI Ethics Framework have instituted new methods of protection, which incorporate elements of transparency and fairness. The Australian Human Rights Commission Initiative incorporates the same principles. Each of these developments highlights the necessity of regulatory practices toward the use of AI technologies in the informational ecosystem.
Despite these active attempts, the limitations are clear. The speed of technological innovation is faster than the regulator’s control. The balance between the owner of the platform and the need for public transparency is still missing. Many systems are located in various places on the globe and exist outside the control of any one government. Prioritization of profit maximization and user engagement over safety, equity, and honesty has proven common among corporations. Current methods of artificial intelligence handling have indicated neither comprehensive and purely constraining systems, or effective control of automated systems. The only means of achieving true reform is through laws instead of guidelines. Laws will only be effective with true transparency and involvement of the affected populations.
Automated systems attempt to improve overall processes, but governance must still retain control. Technology should be rule-based to enforce control and governance. The existing systems that these technologies impact must work. Regardless of the level of impact on the systems, governance must ensure public advocacy and adjustment to control. Governance is not routinely replacing systems, but embedding control and responsibility into systems, and that means that when systems malfunction, the people running the systems are still accountable.
Technology Should Serve People
We should neither fear artificial intelligence nor accept it without scrutiny. AI is both a reflection of social injustices and a social construct that requires managing. People, especially those concerned with equality, should not accept inaction as AI constructs maladaptive, social-determining decisions. We need to demand accountability from the decision-makers, both governmental and corporate, to ensure that we understand the processes behind decision-making, we have the right to dispute decisions when they are unfit, and to explain the processes behind decisions.
The primary focus should not be on maintaining control over a sociotechnical system, instead, focus should be on the underlying power structure, so that the power that governs the system is answerable to the populace who are affected by it. We are living in a time when one thing should be clear: Technology is to serve the people, not the other way around.
Reference List
Crawford, K. (2021). Atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.
European Commission. (2024). The AI Act. European Commission.
https://commission.europa.eu/topics/artificial-intelligence_en
UNESCO. (2021). UNESCO Recommendation on the Ethics of Artificial Intelligence.
Australian Human Rights Commission. (2024). AI and human rights initiatives. Australian Government.
Be the first to comment