A look at OpenAI's open-source speech recognition software Whisper, which can transcribe speech in more than 90 languages, outperforming humans in some of them
OpenAI's open-source speech-transcription program—that shows us where machine learning is going. https://www.newyorker.com/... @niemanlab : “Ever since I've had tape to type up—lectures to transcribe, interviews to write down—I've dreamed of a program that would do it for me.” https://www.newyorker.com/...
Context & Ripple Effects
When OpenAI open-sourced Whisper on 680K hours of web audio, the pitch was accuracy at human-or-better levels across 90+ languages — and this New Yorker piece is the early-2023 portrait of what that unlocked for anyone with tape to transcribe. It reads now as the optimistic midpoint of a longer arc.
That arc has since bent twice: sources reported OpenAI used Whisper to transcribe over a million hours of YouTube videos as GPT-4 training text, and engineers later documented that the same model hallucinates chunks of text, including racial commentary. The profile captures the tool at peak promise, before both the data-provenance questions and the reliability questions surfaced.
First-order effects
- Journalists, researchers, and developers get production-grade multilingual transcription for free, displacing paid dictation and manual transcribing workflows overnight.
Second-order effects
- Whisper becomes dual-use infrastructure: the same open weights that power hobbyist transcription also serve as a bulk data-harvesting pipeline for training frontier models like GPT-4.
Third-order effects
- As voice moves into commercial APIs — OpenAI's later launch of GPT-Realtime-Whisper alongside reasoning and translate models shows transcription absorbed into a broader voice stack — the hallucination findings raise the stakes for error-tolerance standards wherever transcripts feed decisions.
The trend: Speech recognition is being commoditized from standalone product into a reusable component of larger AI pipelines, with accuracy claims and provenance questions trailing behind adoption.