“Mum, I have been kidnapped. please send $200,000 and save me…”
Figure 1: Scammers usually speak in Mandarin and claim to represent a Chinese authority. Picture NSW Police, https://www.news.com.au/national/crime/alarm-raised-over-virtual-kidnapping-scams-targeting-international-students/news-story/da63fdf52c34b58ccd54e010cbc6d2c4
AI as a tool aiding cybercrime
In recent years, Australian police have been on high alert as a wave of insidious “virtual kidnapping” scams, orchestrated by sophisticated transnational criminal rings, using artificial intelligence as a tool to aid in cybercrime. They target vulnerable international students across the country, the majority with Chinese backgrounds. In October 2023 alone, the New South Wales Police State Crime Command’s Robbery and Serious Crime Squad has already uncovered three separate incidents of virtual kidnappings. These scams have left families desperate, with perpetrators extorting hundreds of thousands of dollars and demanding staggering sums exceeding $1,000,000 from distraught loved ones under the guise of ransom payments.
Where the crime begins
The beginning of these scammers starts with a phone call, typically conducted in Mandarin, where they impersonate Chinese authorities such as the Embassy, Consulate, or Police. Victims are coerced into believing they are embroiled in criminal activities in China or have fallen victim to identity theft, with scammers insisting that a hefty fee must be paid to forestall imminent legal repercussions, arrest, or deportation. Employing sophisticated tactics, the perpetrators manipulate victims into providing staged photos and videos depicting their purported abduction and confinement. Leveraging the advancements in artificial intelligence, voice cloning technology is utilized to replicate the victim’s voice, enabling scammers to contact their families in China and perpetuate the illusion of an urgent and desperate situation. These fabricated communications often include simulated cries and screams, or deepfake videos portraying the victim in distressing scenarios, further compelling families to comply with ransom demands.
Targeted groups
These fraudulent schemes primarily focus on adolescents and young individuals, taking advantage of their vulnerability and warning them of potential harm to their families in China to ensure compliance.
” Despite many of the students having read the brochures, they could still fall victim, these scams are very, very convincing. If a student is young and vulnerable, they’re targeted, they don’t have a support network here in New South Wales”—- Detective Superintendent Doueihi(ABC News, October 2023).
The use of AI voice cloning
In some instances, scammers escalate their tactics by harvesting voice biometrics from the victim’s social media posts or utilizing deepfake technology to mimic the voices of movie actors, adding layer of authenticity to their deceitful communications. As Australian authorities race to combat this alarming trend, it underscores the urgent need for heightened vigilance and proactive measures to protect unsuspecting individuals from falling prey to these insidious cybercrime schemes.
Figure 2: Coote, G. (2023). NSW Police have received multiple reports of so-called “virtual kidnapping scams.(Supplied: NSW Police). ABC News. photograph. Retrieved April 4, 2024, from https://www.abc.net.au/news/2023-10-18/nsw-digital-kidnapping-targets-chinese-students-in-sydney/102993076
What is AI Voice clone?
Figure 3: How cybercriminals can perform virtual kidnapping scams using AI voice cloning tools and chatgpt. Security News. (n.d.). https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/how-cybercriminals-can-perform-virtual-kidnapping-scams-using-ai-voice-cloning-tools-and-chatgpt
Voice cloning scams, also known as deepfake scams, are a growing threat to businesses as well as individuals. These scams involve the use of artificial intelligence (AI) and machine learning (ML) to convincingly imitate someone’s voice in order to trick victims into revealing sensitive information or transferring funds (Pérez et al.,2021), while also, as previously mentioned, forging the victim’s identity. Voices come to deceive families into paying ransom, fueling online kidnapping cases.
This is accomplished through deep learning AI speech generators, it is advanced software that mimics human speech effectively (Benois-Pineau,2023). Deep learning is a machine learning method based on artificial neural networks (Benois-Pineau,2023). By utilizing these neural networks, artificial intelligence can process data in the same way that neurons in the human brain interpret information. This indicates, the more human-like an AI is, the better it can imitate human behavior Pérez et al.,2021). In other words, the more speech data they are exposed to, the better they become at imitating human speech. This is how artificial intelligence works by capturing any person’s voice and weaponizing it, mimicking all the rhythm, speech patterns, and intonation of the cloned individual.
Around the world
Figure 4: The moment authorities find ‘cyber kidnapped’ teen, BBC News, https://www.bbc.com/news/world-us-canada-67861852
While the case of cybercrime raises concern in Australia, on the other side, Singapore and America are also facing the same challenge. In 2nd January 2024, shocking news of 17-year-old international student Kai Zhuang was reported missing. After two days of searching, police finally found him alive, he was discovered very scared and cold in a tent in rural Utah. He was threatening to isolate himself over there, and take photos of himself being ‘kidnapped’, otherwise his family would get hurt. Kidnappers use the photos to extort his family for money, alongside AI voice- clone tools to make his parents believe his suffer, they use fake voice clips such as screaming and distress to convince the family that the victim is in trouble and needs to pay up first. Kai Zhuang’s parents were tricked into paying $80,000 us dollars. While in Singapore, there were several similar cases as well, all targeted at international students living there without parents (Lam, 2024).
The accessibility of AI voice clone
Figure 5, a screenshot of the search ‘AI voice clone’.
AI voice cloning has become quite accessible for internet users, with a wealth of resources and tools readily available on platforms like Google with just a simple search. While it is convenient for legitimate purposes such as dubbing and voice-over work, it also creates significant potential hazards, such as boosting cybercrime. With just a few clicks, criminals can collect audio samples and deploy sophisticated algorithms to generate cloned voices, allowing them to manipulate victims and orchestrate elaborate scams with surprising ease. The widespread use of AI voice cloning therefore highlights the urgent need for strong cybersecurity measures and increased awareness to mitigate the risk of exploitation and protect individuals from falling victim to these malicious activities.
Figure 6: ElevenLabs: Ai Voice Cloning: Clone your voice in minutes. ElevenLabs. (n.d.). https://elevenlabs.io/voice-cloning
For instance, as demonstrated by the screenshot of Eleven Labs, which boasts a high Google review rating and advertises ‘voice cloning capabilities in minutes’, the accessibility of such technology demands a proactive approach to mitigate its potential for exploitation.
Figure 7 Elevenlab’s statement on its legal regulations. (April 2024).
As depicted in Figure 6, Eleven Labs asserts a policy against the cloning of voices belonging to political figures, known as the “No-Go voice policy,” aimed at curtailing the creation of deceptive content. Nevertheless, this policy appears to overlook the potential threat posed by voice cloning for cyber-kidnapping, indicating a need for greater emphasis on addressing this specific concern.
Potential solution
AI, with three key pillars: ‘algorithm’, ‘hardware’ and ‘data’ is profoundly impacting our contemporary society. By collecting large amounts of data, using machine learning (ML), and learning to find inter-dependencies among these different data, they learn on every new piece of data they meet’ (Megorskaya, 2022).
There is a pressing requirement to allocate more attention and resources toward mitigating the risks associated with voice cloning in the context of cyber-kidnapping. Consequently, the development of an innovative framework might be able to cope with these challenges effectively.
The Ethics of Artificial intelligence in a global scale
Figure 8: Unesco. (2024, January 1). Global Forum on the ethics of AI 2024. Global Forum on the Ethics of AI 2024. https://www.unesco.org/en/forum-ethics-ai
According to Visvizi & Bodziany (2021), The ethics of warfare with the use of artificial intelligence (AI) can be considered at two levels. The first—technological—is associated with the human’s endeavor to achieve a higher development level in all areas of life. One of them is the military sphere, including the development of military technologies necessary to provide the state not only with guarantees of domestic security but also with “national power” or, in a broader context, with state power. The second level concerns the axiological sphere in which “applied ethics” is located, focusing on the practical aspects of applying AI in everyday life. daily, we should consider thoroughly how to ‘govern’ AI and how might AI subtly ‘govern’ us. In February 2024, UNESCO just held a 2nd Global Forum on the Ethics of AI: Changing the Landscape of AI Governance. It mentioned the rapid rise in artificial intelligence (AI) has created many opportunities globally, from facilitating healthcare diagnoses to enabling human connections through social media and creating labor efficiencies through automated tasks.
However, these rapid changes also raise profound ethical concerns. These arise from the potential AI systems must embed biases, contribute to climate degradation, threaten human rights, and more. Such risks associated with AI have already begun to compound on top of existing societal inequalities, resulting in further harm to already marginalized groups.
Therefore, there is an urgent need for AI governance. We are calling for mutual learning based on the lessons and good practices emerging from the different jurisdictions around the world.
AI governance
Figure 9: Giardino (2020). The mirage of a global framework for AI governance. Medium. https://medium.com/carre4/the-mirage-of-a-global-framework-for-ai-governance-35b88a36615c
From a global lens of AI governance, there are diverse approaches that reflect the complex interplay of technological advancement and societal values. In the United States, governance remains patchwork, with a strong emphasis on anti-discrimination and affirmative action laws, though efforts towards comprehensive regulation are ongoing. Europe stands out for its stringent privacy laws, notably the General Data Protection Regulation (GDPR), and is pioneering a risk-based approach with its proposed ‘first-ever legal framework on AI.’ Singapore adopts a balanced approach, as seen through its AI governance framework and initiatives such as the Trusted Data Sharing Framework and intellectual property protections. At the international level, organizations like UNESCO strive to create a cohesive global policy framework. Transitioning to Australia, the focus shifts to the recommendations of the Human Rights Commission, indicating a growing recognition of the need for robust AI regulation within the country’s legal framework. As nations grapple with the complexities of AI governance, Australia is poised to contribute its unique perspective, potentially shaping a nuanced regulatory landscape tailored to its socio-political context.
AI governance in Australia
Figure 10: Our work to combat virtual kidnappings. AUSTRAC. (2024). https://www.austrac.gov.au/our-work-combat-virtual-kidnappings
In Australia, we are practicing Human Rights Commission recommendations as a regulation for AI. According to Human Rights and Technology Final Report (2021), self-regulation and co-regulation, alongside with law that upholds human rights are expected to be promoted as effective regulation.
However, according to ASIC chair Joe Longo on 31st January 2024, “our current regulation around AI may not be sufficient”. Existing laws likely do not adequately prevent AI-facilitated harms before they occur, and more work is needed to ensure there is an adequate response to harms after they occur.”
Joe mentioned, “We need to think about, what would need to be addressed for the regulatory framework to ‘fit the bill’? Or, to put it another way: in what way might the current regulatory framework inadequately prevent AI-facilitated harms? This question is key. We can only bridge the gap – and create our best approximation to the ideal – if we know where that gap lies.”
Conclusion
Considering the alarming cases in virtual kidnapping scams targeting vulnerable international students, it is evident that the convergence of artificial intelligence (AI) and cybercrime presents a strong challenge for law enforcement agencies worldwide. The sophisticated tactics employed by criminal rings underscore the urgent need for enhanced vigilance and proactive measures to safeguard individuals from falling victim to these insidious schemes. The accessibility of AI voice cloning tools further exacerbates the threat, highlighting the imperative for robust cybersecurity measures and increased awareness to mitigate the risk of exploitation. Furthermore, the ethical implications of AI governance in the context of cybercrime necessitate a comprehensive regulatory framework that upholds human rights and addresses emerging challenges effectively. As nations grapple with the complexities of AI governance, collaborative efforts are essential to foster mutual learning and develop nuanced regulatory frameworks tailored to the socio-political context of each jurisdiction. In Australia, while strides have been made in aligning regulatory efforts with human rights principles, there remains a pressing need to bolster existing laws and regulations to effectively combat AI-facilitated harms. By addressing these challenges collectively and leveraging innovative solutions, we can strive toward a safer and more secure digital landscape for all.
Reference
Benois-Pineau, J. (2023). Explainable deep learning AI : methods and challenges. Academic Press.
Crawford, K. (2021). The Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence (1st ed.). Yale University Press. https://doi.org/10.2307/j.ctv1ghv45t
Djenna, A., Barka, E., Benchikh, A., & Khadir, K. (2023). Unmasking Cybercrime with Artificial-Intelligence-Driven Cybersecurity Analytics. Sensors (Basel, Switzerland), 23(14), 6302-. https://doi.org/10.3390/s23146302
Giardino, E. (2020, August 1). The mirage of a global framework for AI governance. Medium. https://medium.com/carre4/the-mirage-of-a-global-framework-for-ai-governance-35b88a36615c
Healey, J. (Ed.). (2020). Artificial intelligence. Spinney Press.
Lam, N. (2024, January 11). Explainer: what is a “cyber kidnapping” scam, and how can you protect yourself against it? Today. Retrieved 2024, from https://www.todayonline.com/singapore/explainer-cyber-kidnapping-scam-protect-2336491.
Megorskaya, O. (2022, June 27). Training Data: The Overlooked Problem Of ModernAI. Forbes. https://www.forbes.com/sites/forbestechcouncil/2022/06/27/training-data-the-overlooked-problem-of-modern-ai/?sh=5cbf89a9218b
Pérez, A., Díaz-Munío, G. G., Giménez, A., Silvestre-Cerdà, J. A., Sanchis, A., Civera, J., Jiménez, M., Turró, C., & Juan, A. (2021). Towards cross-lingual voice cloning in higher education. Engineering Applications of Artificial Intelligence, 105, 104413-. https://doi.org/10.1016/j.engappai.2021.104413
Safe and Responsible AI in Australia Consultation: Australian Government’s Interim Response, p. 5
Unesco. (2024, January 1). Global Forum on the ethics of AI 2024. Global Forum on the Ethics of AI 2024. https://www.unesco.org/en/forum-ethics-ai
Visvizi, A., & Bodziany, M. (Eds.). (2021). Artificial Intelligence and Its Contexts Security, Business and Governance (1st ed. 2021.). Springer International Publishing. https://doi.org/10.1007/978-3-030-88972-2
Be the first to comment