A profile of ElevenLabs, whose founders seem unprepared for how its impressive AI voice cloning tech can change the internet and unleash political chaos
My voice was ready. I'd been waiting, compulsively checking my inbox. I opened the email and scrolled until I saw a button that said, plainly, “Use voice.”
This profile matters because it shifts attention from the capability itself to whether the company building it is equipped to manage the political and social consequences of scaled voice imitation.
First-order effects
ElevenLabs faces sharper scrutiny over its safeguards, governance, and readiness to curb harmful voice cloning, particularly where political impersonation is concerned.
Users, platforms, and audiences must treat apparently authentic audio with more caution as convincing cloned speech becomes easier to produce.
Second-order effects
Voice-cloning providers, including firms offering tightly limited products such as OpenAI’s partner-only Voice Engine, face pressure to distinguish their access controls and misuse protections.
Political campaigns, media organizations, and services relying on voice identity have stronger incentives to add verification and provenance checks rather than trust audio alone.
Third-order effects
If synthetic voices become broadly available faster than authentication norms mature, audio may lose some of its standing as standalone evidence of identity or intent.
The sector’s competitive advantage could increasingly depend on consent, traceability, and enforcement systems—not only voice quality—as misuse risks invite more institutional scrutiny.
The trend: Generative-AI voice tools are moving from novelty products toward trust-critical infrastructure, making safety controls and verification central to their adoption.
“Early on, ElevenLabs' model began randomly inserting applause breaks after pauses in its speech: It had been training on audio clips from people giving presentations in front of live audiences.” So many good details in @cwarzel's feature on ElevenLabs https://www.theatlantic.com…
@iethics
@iethics on x
“An ElevenLabs voice might be reading this article to you: The Atlantic uses the software to auto-generate audio versions of some stories, as does The Washington Post”: https://www.theatlantic.com/ ... #ethics #AI #tech #voice
@cwarzel
Charlie Warzel on x
i wrote about elevenlabs, the voice ai company whose tech i find probably the most impressive of all the gen ai tools but also the weirdest. my feeling is that each one of these tools is a chaos agent, that will hook in with other chaos agents in unpredictable ways. it's...
@saahil_desai
Saahil Desai on x
“I typed some words into a text box, hit Enter, and there I was: all the nasal lilts, hesitations, pauses, and mid-Atlantic-by-way-of-Ohio vowels that make my voice mine.” Fascinating piece by @cwarzel on ElevenLabs and voice cloning https://www.theatlantic.com/ ...
@jadubin5
Yaya Dubin on x
Any time I read about one of these companies, the only thing I can think about is, why do they even want to create this? The range of good outcomes from this technology is so small & the range of bad outcomes is basically infinite. They know it shouldn't exist but make it anyway.
@theatlantic
@theatlantic on x
ElevenLabs can clone basically anyone's voice in a few hours. @cwarzel on how nobody, not even its creators, seems ready for its chaotic potential: https://www.theatlantic.com/ ...
@iethics
@iethics on x
“Soon, I had high-quality audio of my voice clone urging people not to vote, blaming ‘the globalists’ for COVID.... It was enough to make me check with my bank to make sure any potential #voice-authentication features were disabled.” #ethics #AI #disinformation #privacy #business
@scottnover
Scott Nover on x
Agree that ElevenLabs is the most impressive genAI tool I've tried. That or Midjourney.