system protects and sacrifices
users simultaneously.
When you open Facebook and find a violent post calling for action against you one day. So you report them. Unfortunately, there is no response. And then, you report it over and over again, but there is no one result at all. Meanwhile, the posts keep spreading so that the situation get worse. For millions of users living in Myanmar, the Philippines, India, and Indonesia, this situation is actual situation. It happens every day. We hear a lot about AI being the solution to resolve online hate speech. Platforms like Meta invest billions of dollars in artificial Intelligence systems that can review content in a huge scale. It can flag and remove harmful content before it spreads. It seems that great progress has been made. However, the actual situation is more complicated. AI systems are not neutral. They are shaped by the priorities, resources, and limitations of the companies that build them. And now, those limitations affect users in non-English speaking countries injusticely. The platform can efficiently review content by artificial intelligence. Meta can process more than 100 billion content every day. This produces a huge labor cost in need. That’s why the platform has implemented automated systems and trained algorithms to detect harmful content and remove harmful speech. From a governance perspective, AI makes a great sense. It is faster and cheaper than human review. It applies the same rules to everyone. Platform governance is effectively “lawless,” not because rules are absent, but because they are created and enforced without external accountability (Suzor, 2019, pp. 16–17). The Meta platforms are constantly evolving and training AI models to meet the needs of users with data. Meta reported in early 2021 that an artificial intelligence system automatically detected 97.1% of hate speech on Facebook (Meta, 2021). This makes great progress. But the result raises an important question.
The 97.1% data does not explain which languages was used to express hate speech.
AI content regulation relies on training data. More available data in specific language used to training will lead to a better performance in detecting harmful content. English usually has the largest data in the market, where a self-reinforcement cycle appears, in which detecting hate speech in the English community will become increasingly effective; however, in other language environments, the opposite is true.
A case study from researchers at the University of Sydney and the University of Queensland. Facebook: Regulating Hate Speech in the Asia Pacific shows this imbalance. Studying Facebook’s moderation practices across the following countries: India, Myanmar, Indonesia, the Philippines, and Australia, the researchers found that hate speech is hard to detect, when posted in minority languages, regional dialects, or local slang (Sinpeng et al., 2021, pp. 37–40). LGBTQI+ groups in India, Indonesia, and the Philippines reported that being exposed to high levels of discriminatory and threatening content that Facebook’s automated filters failed to detect.
The researchers also found that when they requested access to Facebook’s dataset of removed hate speech to evaluate moderation performance, the company declined, because of privacy (Sinpeng et al., 2021, pp. 37–40). This highlights a broader issue. Platform governance is under their controlling where platforms build model and evaluate it. There should be more external governance (Suzor, 2019, pp. 16–17).
A clear example of these challenges can be seen in Myanmar. During the 2010s, Facebook became the internet for many Burmese users effectively. Facebook can access free data through local telecom providers. Facebook is becoming an Internet substitute for most public in Myanmar. The platform significantly influences public opinion at a politically volatile moment. For most people, the platform became the majority source of news, communication, and community. However, as Facebook’s user base in Myanmar increases rapidly, investments in Burmese-language moderation can not catch the pace.
In 2014, Meta had only one Burmese-speaking content moderator in Dublin. This staff was responsible for more than one million users’ posts (Amnesty International, 2022, pp. 7–8). The same year, a fake post said two Muslim men has connection with rape, which stimulated violent riots in Mandalay. These warning signs were significant. Whereas internal documents later displayed that by 2020, only 2% of hate speech had taken measures against (Amnesty International, 2022, pp. 7–8).
The consequences were severe. Facebook’s algorithms did not just fail to remove anti-Rohingya content but also spread it. Internal Meta documents revealed that this platform was designed to increase engagement, which makes hate speech spread under the same algorithms (Amnesty International, 2022, pp. 7–8). The violence made more than 700,000 Rohingya people homeless, leading Amnesty International to conclude that Meta made serious human rights violations (Amnesty International, 2022, pp. 7–8).
This example reflects an argument made by Kate Crawford in Atlas of AI (2021). Crawford argues that AI model is shaped by many factors, for example, economic, political, cultural, and historical forces. In her words, artificial intelligence plays a “registry of power.” AI systems are created to serve those humans who have already held power (Crawford, 2021, pp. 8–9). When Facebook’s algorithms pushed content, the system do not consider what content is it, it just focus on the result which is the function of the algorithms.
Myanmar’s events reveal a problem. Platforms prefer to invest more in developed regions. However, developing regions with large numbers of users often receive less AI support and less attention. This gap creates inequality.
We should ask not only whether AI systems work or not, but also why this AI model is designed, who the stakeholders are, and who can control this tool (Crawford, 2021, pp. 8–9). In contemporary times, the answers often favour English-speaking users in wealthier markets. In a sense, language has become a form of power. Sinpeng et al. (2021) found that Facebook used only one global policy across different cultural and languages with few changes. As the researchers stated, “hate speech needs to be controlled under local language conditions and cultural knowledge.” (Sinpeng et al., 2021, p. 39). In other words, a word or phrase that can slander someone may not be detected under English conditions. In addition, a report of hate speech is usually ignored; mostly, page administrators said “they received automated messages and no follow-up,” and were simply told the content “did not meet the criteria as set out in Community Standards”, even the harmful content is obvious (Sinpeng et al., 2021, p. 38).
The AI tool, such as ChatGPT, Gemini, is based on a concept from Computer Science called Large Language Model (LLM). LLM is a model that can generate text according to input text. Originally, it is just a higher model like other deep learning models it is not as powerful as what we have used. Having been developed for several years, we can make LLM use a function to browse web pages, identify and understand images and videos. Moreover, based on the RAG technique, the accuracy of LLM replies has improved. Combined all the techniques and functions, LLM becomes the AI we know with unbelievable efficiency for tasks (Zhao et al., 2023).
A common argument is that imperfect AI moderation is better than no moderation at all. While this is partly true, it misses a key issue: accountability.
Platforms have the power over all content in their own systems currently. Suzor argues that platform governance is effectively “lawless,” not because rules are absent, but platforms make all rules without external regulation and suggestion (Suzor, 2019, pp. 16–17). When researchers request data, access is often denied (Sinpeng et al., 2021, p. 37). When users appeal decisions, they frequently receive automated responses with no follow-up (Sinpeng et al., 2021, p. 38). When transparency is requested, platforms usually publish a wide range of macro statistics without language-specific breakdowns, for example, Meta’s claim that AI detected 97.1% of hate speech (Meta, 2021). But if no one can see what is really happening, nothing will change.
Algorithms are not enough to improve content review. Greater transparency is essential. Platforms should publish review data in different languages in detail so that researchers and regulators can assess regional differences. Moreover, working with the local community can also be effective. As Sinpeng et al. (2021, pp. 37–40) said, platforms should spend more attention on local-language moderation before harm happens, not after. Crawford (2021, pp. 8–9) reminds us that AI systems are created for certain interests. The real question is not whether AI works. It is whether it works for the public or some specific group.
The next time a platform reports that AI detects 97.1% of hate speech, we had better ask: 97.1% in which language? These are not small questions. The real issue is who these systems serve to protect. Unfortunately, these platforms are willing to invest more in the market to obtain more benefits. English-speaking users in developing region usually get better protection than others. The events in Myanmar show the results of ignoring this gap. There is no possibility for one moderator to handle hate speech among approximately one billion users. A genocide that Amnesty International concluded Meta had contributed to through its failure to act (Amnesty International, 2022, pp. 7–8). These are the results of ignoring the needs of those users.
The problem does not stop at Myanmar. Similar problems still strike across minority communities in India, Indonesia, and the Philippines, where data access is denied, reports of harmful speech is replied by program without follow-up (Sinpeng et al., 2021, pp. 38, 40).
This is not only a technology problem. It is a governance problem. Platforms stand on top of the right without regulation. Similar problems will appear continuously in different countries until platforms changed. Things can get better, but only if platforms are willing to make real changes. Platforms need to be held accountable by the external organisations. They need to publish data by different language regions, instead of a vague total number. They need to enhance local moderation system avoiding harmful public opinion.
The language you speak should not decide how safe you are online. But right now, for billions of people, it does.
Amnesty International. (2022). The social atrocity: Meta and the right to remedy for the Rohingya. https://www.amnesty.org/en/documents/asa16/5933/2022/en/
Crawford, K. (2021). Atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press.
Meta. (2021, February 11). Update on our progress on AI and hate speech detection. https://about.fb.com/news/2021/02/update-on-our-progress-on-ai-and-hate-speech-detection/
Sinpeng, A., Martin, F. R., Gelber, K., & Shields, K. (2021). Facebook: Regulating hate speech in the Asia Pacific. Department of Media and Communications, The University of Sydney. https://doi.org/10.25910/J09V-SQ57
Suzor, N. P. (2019). Lawless: The secret rules that govern our digital lives. Cambridge University Press.
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Liu, P., Nie, J.-Y., & Wen, J.-R. (2023). A survey of large language models. arXiv. https://arxiv.org/abs/2303.18223
Be the first to comment