How do AI scams exploit empathy for financial gain?

Question

Accepted Answer

AI scams exploit empathy by using fake disability narratives to push people to buy from suspicious shops
AI-generated videos that appear to show people with Down syndrome are used to drive product sales by targeting viewers’ sympathy. Across TikTok, YouTube, and Instagram, the AI characters depict themselves as being bullied for selling crafts while using insults and disparaging language about people with disabilities, and the clips then link to online storefronts that direct users to buy resin lamps, crochet handbags, and clay bowls. Advocates say the approach reinforces the idea that people with Down syndrome “need to be pitied,” turning that sentiment into a sales funnel.
They stage “bullying” stories while selling crafts through linked storefronts
The videos commonly show AI characters claiming they are being bullied for selling items, paired with insults and disparaging language about people with disabilities. Instead of directing viewers to established sellers, the posts link to suspicious online storefronts, including one listing multiple “five-star reviews” that all use the same filler text. Investigators also found that some product designs had previously appeared in real creators’ content before being appropriated.
They rely on stereotypes that viewers feel compelled to respond to
Nathan Rowe of Down Syndrome International said the videos play into a stereotype that people with Down syndrome, caused by an extra chromosome, should be pitied. He described the tactic as preying on viewers who hold a sympathetic but paternalistic view of the condition. In his account, the scams also “crowd-out” posts from actual entrepreneurs with Down syndrome, potentially diverting business away from real sellers.
They use easily generated fakes and repeatable tactics to scale quickly
An AI researcher, Jeremy Carrasco of the firm Riddance, said the large number of accounts promoting this kind of content points to a profit-making scheme. He attributed the spread to “system-wide failures,” adding that the videos are exceedingly easy to create and difficult to track at scale. Carrasco also noted similar patterns, including earlier attempts to sell identical products using elderly synthetic characters.
They fit a broader pattern of identity theft and deception across platforms
Down Syndrome International previously complained to Meta about sexualized deep-fakes involving people with Down syndrome, which led to the removal of many videos. While platforms including TikTok and YouTube prohibit deceptive or manipulative activity and misleading spam, enforcement is challenging, and other accounts can continue reposting the AI clips even after some are taken down. AFP reported that some of the pottery products and designs used in the videos were found in earlier real-world listings, including instances where a crochet bag design appeared on Shein, a site associated with dropshipping storefronts.

Reading history

Hide →

This history is cleared when the page is closed.

How do AI scams exploit empathy for financial gain?

Loading

Was this article helpful?

Helpful

Not helpful

Report an issue

1

0

0

Share with friends