The Weird World of AI Voice Replicas
This is Atlantic Intelligence, a limited-run series in which our writers help you wrap your mind around artificial intelligence and a new machine age. Sign up here.
It’s a good bet that the generative-AI era will be stranger than anyone expects. In a new feature for The Atlantic, my colleague Charlie Warzel profiles ElevenLabs, an AI company that specializes in replicating voices.
“It’s easy, when you play around with the ElevenLabs software, to envision a world in which you can listen to all the text on the internet in voices as rich as those in any audiobook,” Charlie writes. “But it’s just as easy to imagine the potential carnage: scammers targeting parents by using their children’s voice to ask for money, a nefarious October surprise from a dirty political trickster. I tested the tool to see how convincingly it could replicate my voice saying outrageous things. Soon, I had high-quality audio of my voice clone urging people not to vote, blaming ‘the globalists’ for COVID, and confessing to all kinds of journalistic malpractice. It was enough to make me check with my bank to make sure any potential voice-authentication features were disabled.”
You may have already encountered ElevenLabs’ technology without realizing it. The Atlantic and The Washington Post use the software to produce audio versions of some stories. Nike cloned the NBA star Luka Dončić’s voice with the software for a recent marketing campaign. New York City Mayor Eric Adams’s office used it to mimic the politician’s voice for multilingual robocalls. And a gun-control nonprofit used it to re-create the voices of children killed in the Parkland school shooting.
The company has established safeguards in an attempt to head off nefarious usage, but it’s reasonable to expect surprises. As Charlie writes, “There are simply too many motivated people constantly searching for ways to use these tools in strange, unexpected, even dangerous ways.”
— Damon Beres, senior editor
ElevenLabs Is Building an Army of Voice Clones
By Charlie Warzel
My voice was ready. I’d been waiting, compulsively checking my inbox. I opened the email and scrolled until I saw a button that said, plainly, “Use voice.” I considered saying something aloud to mark the occasion, but that felt wrong. The computer would now speak for me.
I had thought it’d be fun, and uncanny, to clone my voice. I’d sought out the AI start-up ElevenLabs, paid $22 for a “creator” account, and uploaded some recordings of myself. A few hours later, I typed some words into a text box, hit “Enter,” and there I was: all the nasal lilts, hesitations, pauses, and mid-Atlantic-by-way-of-Ohio vowels that make my voice mine.
What to Read Next
- AI in politics is so much bigger than deepfakes: “Presidential campaigns have long tailored their ads and emails to specific groups. Now any politician can,” Jacob Stern writes.
P.S.
Apple received a ton of blowback this week for a new iPad commercial that, to many viewers, seemed like an uncomfortable reminder of generative AI’s threats to human creativity: It depicts a variety of artistic tools getting crushed in a hydraulic press, leaving a new tablet behind. “Apple wants to show you that the bulk of human ingenuity and history can be compressed into an iPad, and thereby wants you to believe that the device is a desirable entry point to both the consumption of culture and the creation of it … But good Lord, Apple, read the room,” I wrote with Charlie this week. (The company apologized yesterday for the ad.)
— Damon