What are the challenges of AI in detecting online harm?

Question

Accepted Answer

AI Faces Significant Challenges in Detecting Online Harm Due to Linguistic Nuances and Insufficient Human Oversight

Artificial intelligence (AI) systems struggle to accurately detect online harm, particularly cyberbullying, because they often fail to grasp the subtle complexities of human language. This limitation is especially pronounced in languages like Tamil, where AI-based moderation may not effectively identify harmful or abusive content. While social media platforms are increasingly relying on AI for content moderation, a reduction in human moderators exacerbates this issue, as exemplified by TikTok's shift towards AI-driven systems. The Malaysian government is addressing these concerns through the introduction of the Child Protection Code (CPC) and Risk Mitigation Code (RMC), aiming to prevent social media from becoming a breeding ground for crimes such as scams and cyberbullying.
Linguistic Nuances Pose a Barrier to AI Comprehension

AI's inability to fully comprehend the nuances of certain languages presents a significant hurdle in identifying online harm. This is particularly evident with cyberbullying within the Tamil-speaking community, where AI may misinterpret or overlook abusive content that a human moderator would readily identify. The Communications Minister, Datuk Seri Fahmi Fadzil, highlighted this issue, stating that AI cannot be solely entrusted with content moderation due to its limitations in understanding the subtleties of language, especially when it pertains to harmful or bullying statements.
The Trade-offs Between AI and Human Moderation

The increasing reliance on AI for content moderation, while potentially offering scalability, comes at the cost of human oversight. TikTok, for instance, has reduced its human moderator count in favour of AI, a strategy that raises concerns about the thoroughness of content review. The minister's statement suggests a concern that this shift could lead to a rise in undetected cyberbullying, as AI lacks the contextual understanding and empathy that human moderators bring to their work. The tragic case of influencer Esha, who died on July 5, 2024, after reporting threats and defamation, underscores the critical need for effective content moderation, whether AI-assisted or human-led.
Government Initiatives to Mitigate Online Harms

In response to these challenges, the Malaysian government has introduced the Child Protection Code (CPC) and Risk Mitigation Code (RMC) for social media platforms. These codes are designed to ensure that platforms do not become conduits for criminal activities like scams and cyberbullying. The Malaysian Communications and Multimedia Commission (MCMC) is actively engaging with platforms, including YouTube in Singapore, to facilitate the implementation of these codes. Platforms have been granted several months to study and implement the directives, with ongoing discussions regarding their operational framework.
Penalties for Non-Compliance and Age Verification Measures

The implementation of the CPC and RMC is coupled with stringent enforcement measures. Social media platforms are also introducing age verification through electronic Know Your Customer (e-KYC) mechanisms. Once the CPC and RMC are fully implemented, platforms that fail to comply will face substantial penalties. Furthermore, the government advocates for parental involvement in online child safety, including educating children about cyberbullying and scam tactics, being aware of AI-related risks, and utilising parental controls. The recommendation remains that children under 16 should not independently open social media accounts, though parents may do so for educational purposes.

Reading history

Hide →

This history is cleared when the page is closed.

What are the challenges of AI in detecting online harm?

Loading

Was this article helpful?

Helpful

Not helpful

Report an issue

2

0

0

Share with friends