Impossible Content Moderation Dilemmas: Talking About Racism Blocked As Hate Speech

For all the completely evidence-free talk of "anti-conservative bias" on social media, as we keep pointing out, the real problem is that moderating content at scale is impossible to do well. I know I've been repeating this a lot lately, but it's because some people still don't seem to be getting this, or why it's important.

Take, for example, this recent USA Today story talking about the content moderation woes not of conservatives on the platform, but of black users trying to talk about racism on Facebook. Or women talking about sexism. It's getting blocked as hate speech.

For Wysinger, an activist whose podcast The C-Dubb Show frequently explores anti-black racism, the troubling episode recalled the nation's dark history of lynching, when charges of sexual violence against a white woman were used to justify mob murders of black men.

"White men are so fragile," she fired off, sharing William's post with her friends, "and the mere presence of a black person challenges every single thing in them."

It took just 15 minutes for Facebook to delete her post for violating its community standards for hate speech. And she was warned if she posted it again, she'd be banned for 72 hours.

This kind of thing apparently happens all the time.

She says black people can't talk about racism on Facebook without risking having their posts removed and being locked out of their accounts in a punishment commonly referred to as "Facebook jail." For Wysinger, the Neeson post was just another example of Facebook arbitrarily deciding that talking about racism is racist.

"It is exhausting," she says, "and it drains you emotionally."

Of course, this kind of thing happens all the time and we've been pointing it out for years. Rules against hate speech are used to suppress the oppressed talking out about oppressors. Rules against terrorist content are used to erase evidence of war crimes. Over and over again when you demand that policies be put in place to suppress speech to "protect" people, it's often the people you want to protect who are harmed the most.

Black activists say hate speech policies and content moderation systems formulated by a company built by and dominated by white men fail the very people Facebook claims it's trying to protect. Not only are the voices of marginalized groups disproportionately stifled, Facebook rarely takes action on repeated reports of racial slurs, violent threats and harassment campaigns targeting black users, they say.

While I'm sure it doesn't help matters that white men dominate these companies, I'd argue that we see the same issue with basically every kind of speech suppression effort involving top down rules. Trying to suppress speech about "bad stuff" will always, always, always sweep up important and necessary conversations about the bad stuff -- including people calling out the bad stuff.

That's why it's really time for everyone to start thinking about very different approaches, rather than continuing to insist that if platforms just nerd harder they'll find the magic bullet that allows them to erase "bad content" while keeping all "good content."