When Your Face Is No Longer an Alibi — How Facial Recognition Puts Innocent People Behind Bars

Source：https://stock.xinpianchang.com/photo/details/MRlwL9yPticswOy.html

Have you ever imagined such a scenario: on an ordinary day, the police suddenly show up at your doorstep, telling you that you have committed a crime and they are going to arrest you, and the reason given is that an AI system informed the police that you committed a crime in a place you have never been to! This sounds simply absurd, doesn’t it? However, this has already happened.

Case of Angela Lipps

In July 2025, Angela Lipps, a 50-year-old grandmother, was looking after four children at her home in rural Tennessee when armed police officers showed up at her door. She was accused of bank fraud in North Dakota – a place she claimed she had never visited. The accusation against her was not based on witnesses, fingerprints, or any transaction records, but on a “match result” generated by an AI facial recognition system from a blurry surveillance photo. The photo showed a woman using a fake military ID to withdraw money from a bank over 1,000 miles away. No one called Lipps to ask questions, nor did anyone check her bank records. The AI made the decision – she was arrested (Sottile, 2026).

Lipps spent 168 days in prison. Eventually, with the investigation of a lawyer, Lipps was cleared. Investigation records showed that she had been at her home in Tennessee during the time of the incident. The charges against her were dropped, but she lost her house, car, and dog. The Fargo Police Department admitted to a “mistake” but refused to apologize (Sottile, 2026).

This sounds unbelievable, but similar algorithmic decisions are quietly made every day in the lives of job seekers, insurance applicants, and ordinary people dealing with systems – without any explanation and often without any appeal process. 

The main topic of this blog is that under the characteristic of ‘Black Box’, Facial Recognition Technology has become a serious and growing form of harm for normal people who are subjected to punishment they do not deserve, with no means of challenging the ‘Black Box’ logic behind it, to get rid of this situation, the establishment of a legal framework can no longer wait.

What is “Black Box”?

Source: https://www.shakebugs.com/blog/black-vs-white-vs-grey-box-testing/

Algorithmic systems are like black boxes, an internal logic that is an opaque system to ordinary people (Pasquale, 2015). When people think of the algorithms, they always instinctively assume that they are fair. Crawford (2021), however, argues that algorithms are not impartial tools, they are composed of material resources, human labor, history and political structures, and therefore AI serves the interests of these entities. Gillespie (2014) believes that algorithms can reflect the values of designers or engineers and the priorities of a platform’s (E.g. Google) content. However, as a user, you have no idea how these algorithms work.

‘Algorithmic culture’ refers to the phenomenon whereby algorithmic systems are gradually supplanting cultural intermediaries (e.g. YouTubers, TV Station, etc.), whilst to clarify both contents and users in ways that differ from traditional methods (Hallinan & Striphas, 2016). I’m sure you’ve wondered why you and your friend see different content on the Netflix homepage, even though you both have the same account.

But when the logic of these classifications is hidden, accountability becomes almost impossible. This is the real problem for the ‘Black Box’. When you submit your resume and receive an automatic rejection email with no explanation, or when you file a medical insurance claim and the system rejects it within seconds without giving you any channel to question it. 

The working principle of facial recognition is to treat faces as data and store thousands of faces in the system. You don’t know when your data was collected. You walk on the street, but the database treats you as a suspect. You can’t see which data the system uses, what assumptions it encodes, or whether these assumptions apply to you fairly.

Inside The Black Box- What Went Wrong and Why？

The Lipps‘s case can be regarded as a sample of how the opacity of algorithms can translate into concrete harm in the real world. The police in Fargo, North Dakota, used Clearview AI, a commercial facial recognition tool that collects billions of images from social media, to investigate this fraud case.

Lipps’s case exposed the opacity of Clearview AI – that is, the police departments using the tool have almost no understanding of its working principle, data sources, and facial recognition error rate. Buolamwini and Gebru (2018) found that there were significant differences in the classification accuracy of the algorithm for dark-skinned women, and this error analysis extended to tasks such as facial recognition, identity recognition, and verification, indicating that AI is not 100% accurate. Currently, there is no public evidence that the police department conducted any form of system evaluation before deploying Clearview AI.

This case also revealed the opacity of the police investigation process. Fargo was even unaware that West Fargo had introduced the Clearview AI system. The entire investigation process lacked auditing, cross-checking, and accountability chains, and was overly dependent on AI, causing its role to shift from “assistance” to “decision-making”.

As criminologist Ian Adams told Sottile (2026): “We are adopting new technology so quickly that agencies are left having to rely on promises from vendors of new technology, because no one knows how it actually works. Powerful AI tools can lead to complacency, and investigators must be very careful.” What is even more alarming is that, in the absence of a robust accountability system, ordinary people have no means of protecting their rights or even their own safety.

This Is Not an Accident — It Is Bias Embedded in AI Design

Source: https://www.ogunsecurity.com/post/when-artificial-intelligence-gets-it-wrong-five-cases-that-should-alarm-every-security-professional 

On the surface, the Lipps case appears to be a human error, but in reality, the main problem lies with the algorithm itself.

In Noble’s (2018) book Algorithms of Oppression: How Search Engines Reinforce Racism, she points out something important: AI don’t treat everyone fairly. They harm those who are already vulnerable, and those who have money and social status, these harms are often solvable. As an ordinary people, if you want to challenge AI, you must prove that you have the professional legal knowledge or resources for access AI, and these things are far exceeding the public’s capabilities. 

Lipps is not the first innocent person to be hurt by facial recognition. Many similar cases have happened across the US. And you will notice that all these cases have in common is a lack of regulation and over-reliance, which inevitably leads to ordinary people paying the price to the mistakes.

Outside the Black Box-The Governance Gap

After Lipps was released, the question of accountability is still unresolved. Sottile’s reporting revealed that the Fargo Police Chief acknowledged several errors and announced several corrective measures — monthly reviews of the facial recognition system, cooperation with federal authorities, and discontinuation of the West Fargo AI system. But if you look closely at that Sottile report, you will find that no one apologized to Lipps, no official faced any disciplinary action, and Clearview AI declined to comment. 

This is precisely where the secrecy of the black box cuts off any possibility of accountability. Put simply: when no one knows how a decision was made, no one can be held responsible — because they are not the direct “decision-maker,” but rather, to some degree, an “executor” of the system. 

The EU’s AI Act came into force in 2024, and it represents one of the most serious attempts by any government to regulate artificial intelligence. The Act divides AI systems into four risk levels: unacceptable risk, high risk, limited risk, and minimal risk. This is not just a label, because each level has a very different legal requirement (European Commission, 2026). Certain AI system classified by the EU as high-risk are subject to strict regulation. For example, they are required to operate with high degree of transparency; they must involve human oversight and must undergo assessments to ensure they will threaten anyone’s personal safety or freedom. Clearly, facial recognition falls into high risk, which means such Lipps case is virtually unhappen in the EU.

In contrast with Fargo, the police department did implement some internal adjustments following the Lipps case. However, these adjustments were reactive measures taken after harm had been caused to Lipps, and these changes were not mandated by law. In American society, the approach taken is one of ‘principle of prioritizing innovation’. There is no compulsory test before use, no penalties for misuse. Everything seems a compromise made for innovation. In Australia, AI is governed by the ‘Voluntary Principles on AI Governance’. There are also no mandatory laws or regulations to oversee the development of AI.

By comparison, Australia and the United States lag the EU in terms of AL legislation. Furthermore, the existing regulatory frameworks in both countries fail to provide the public with meaningful protection. The lack of legislation is the primary reason why AI continue to pose serious risks to people’s daily lives.

What Needs to Change?

Source: blog. exeo- digitalsolutions, co. jp 

So what needs to change? Here are four important steps that could make a real difference.

Conclusion

Angela Lipps spent 168 days in prison, losing many precious things without receiving any compensation or apology. This absurd surreal scenario is happening in our real lives. In conclusion, this case exposes the failure of AI that operating as a Black Box and empowered by authority. It also reveals how the lives of ordinary people can be shaped or even completely transformed by new technologies in the absence of robust laws and regulation. This leads us to reflect deeply and created a sense of urgency-establishing a systematic legal framework of AI before more people are harmed has become a important thing that cannot be delayed any longer.

Reference 

Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 77-91. Available from https://proceedings.mlr.press/v81/buolamwini18a.html

Crawford, K. (2021). The atlas of ai: Power, politics, and the planetary costs of artificial intelligence. Yale University Press. 

European Commission. (2026, January 27). AI Act. Directorate-General for Communications Networks, Content and Technology. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai

Gillespie, T. (2014). The relevance of algorithms. Media technologies: Essays on communication, materiality, and society, 167(2014), 167.

Hallinan, B., & Striphas, T. (2016). Recommended for you: The Netflix Prize and the production of algorithmic culture. New media & society, 18(1), 117-137. 

Noble, S. U. (2018). A Society, Searching. In Algorithms of Oppression: How Search Engines Reinforce Racism (pp. 15–63). NYU Press.https://doi.org/10.2307/j.ctt1pwt9w5.5

Pasquale, F. (2015). Introduction: The need to know. In The black box society: The secret algorithms that control money and information (pp. 1–18). Harvard University Press.Sottile, Z. (2026, March 29). Police used AI facial recognition to arrest a Tennessee woman for crimes committed in a state she says she’s never visited. CNN. https://edition.cnn.com/2026/03/29/us/angela-lipps-ai-facial-recognition

Sottile, Z. (2026, March 29). Police used AI facial recognition to arrest a Tennessee woman for crimes committed in a state she says she’s never visited. CNN. https://edition.cnn.com/2026/03/29/us/angela-lipps-ai-facial-recognition