How do algorithms amplify extremist content for vulnerable youth?

Question

Accepted Answer

Algorithms Amplify Extremist Content Through Targeted Content Delivery and Community Reinforcement

Algorithms play a significant role in amplifying extremist content for vulnerable youth by identifying individual interests and preferences to deliver increasingly radical material. This process is often exacerbated by the nature of online communities and platforms, where impressionable young individuals can be exposed to and influenced by violent ideologies. Psychological factors, such as feelings of marginalisation and a desire for belonging, can further increase susceptibility to these online influences.
Algorithmic Identification and Escalation of Extremist Material

Social media algorithms are designed to maximise user engagement by analysing viewing habits and preferences. For vulnerable youth, this can lead to a cycle where algorithms identify a nascent interest in violent or extremist themes, such as those found in "gore" content or first-person shooter games that recreate massacres. The platforms then serve more of this content, creating a feedback loop that progressively exposes the user to increasingly radical material. This was observed in the case of a 14-year-old detainee whose interest in mass shooting games on Roblox led to algorithms pushing him towards content idolising perpetrators of real-life tragedies like the Columbine High School and Pulse nightclub shootings.
The Role of Online Platforms and Communities in Radicalisation

Gaming platforms like Roblox, initially designed for entertainment, can become unintended conduits for radicalisation when they host or link to content depicting violence. Beyond gaming, online true-crime communities and forums can also serve as breeding grounds for extremist ideologies. These communities often foster a shared interest in violence, providing a sense of belonging and validation for individuals who feel alienated. The 14-year-old detainee, for instance, was influenced by both social media algorithms and these true-crime communities, which reinforced his fascination with mass shootings and extremist figures.
Psychological Vulnerabilities and Susceptibility to Online Extremism

Youth who experience feelings of marginalisation, bullying, or personal grievances may be particularly susceptible to extremist ideologies encountered online. These individuals may seek validation or a sense of purpose, which extremist groups can exploit. The detainee's self-reported experiences of bullying and feelings of being a victim contributed to his identification with perpetrators of violence. This psychological vulnerability, combined with exposure to curated extremist content, can lead to the adoption of radical beliefs and intentions, as seen in the drafting of a manifesto by the 14-year-old, which incorporated neo-Nazi symbols and targeted specific groups.

Reading history

Hide →

This history is cleared when the page is closed.

How do algorithms amplify extremist content for vulnerable youth?

Loading

Was this article helpful?

Helpful

Not helpful

Report an issue

1

0

0

Share with friends