Should creators be concerned about Instagram’s AI developments?

On April 22, journalist Emanuel Maiberg of 404 Media published a disturbing investigation into Instagram’s ad ecosystem. The report revealed that Instagram is profiting from multiple ads that explicitly invite people to create nonconsensual deepfake nudes with AI apps. One particularly egregious ad had a picture of Kim Kardashian with text that read, “Undress any girl for free. Try It.” 

These ads ran on Facebook, Instagram, Facebook Messenger, and Meta’s in-app ad network from April 10 to 20. The report is particularly troubling given that deepfakes are an issue specifically impacting teenage girls — a demographic Meta has repeatedly been criticized for harming.

Only a few states have any laws addressing deepfakes. However, Meta reps have said it does ban nonconsensual deepfakes. The company also bans ads containing adult content, saying it uses a mix of AI detection systems and human review to identify this content. Meta deleted these deepfake ads after 404 Media ran its story. But it clearly failed to detect and address them properly on its own.

So why did this happen? Well, as 404 puts it, it’s clear Instagram is either “unwilling or unable” to enforce its own policies about AI. And perhaps that’s because it has some twisted priorities when it comes to the AI tools it is investing in.

How is Instagram enforcing its AI policies?

“Instagram and Facebook are both decaying platforms that don't just enable criminal behavior but actively profit from it and either don't know or don't care how to stop it,” Jason Koebler of 404 Media said on X. “Content moderation is hard. But Meta is at a point now where they are regularly unable to find obvious illegal content unless a journalist sends them a direct link to it.”

This is hardly the first time Meta has failed to enforce its own policies when it comes to AI. Back in March, NBC News found that it was running other ads for sexually explicit deepfake services. One even advertised blurred nude image of an underage actress ...

In Today's Newsletter:

• YouTube Channel Watcher Apologizes After Massive Fan Backlash to Streaming Service Launch
• TikTok Ban-or-Sell Bill Is Now Bundled With Military Aid
• TikTok Lite Faces Legal Scrutiny in the EU
• Social Media Can’t Escape Taylor Swift
• Amazon Dominates the Creator Economy
• What Is the Creator Economy? A Beginner’s Guide To Making a Living Making Stuff

Sign up to receive the Daily Dot’s Internet Insider newsletter for urgent news from the frontline of online.

The post Should creators be concerned about Instagram’s AI developments? appeared first on The Daily Dot.