Sources: OpenAI discussed training GPT-5 on public YouTube video transcripts; AI industry's need for high-quality text data may outstrip supply within two years
Wall Street JournalDeepa Seetharaman
Context & Ripple Effects
The discussion came as OpenAI was preparing its next flagship model; related coverage said some enterprise customers had seen demos of the latest GPT-5 model. The core constraint is not simply access to data, but access to text considered valuable enough for frontier-model training.
Subsequent reporting described OpenAI's large-scale transcription of YouTube videos with Whisper for GPT-4, while Google was also said to have transcribed YouTube material. That places video-derived text at the center of a widening contest for training inputs.
First-order effects
OpenAI's GPT-5 training planning could draw on a substantially broader pool of text derived from public video, with YouTube becoming a strategically important upstream source.
The reported prospect of high-quality text scarcity makes data acquisition and processing more consequential to model-development timelines, alongside model design and compute.
Second-order effects
Model developers and platforms with large archives of video, audio, or text face greater pressure to turn those archives into training-ready datasets rather than leave them as passive content libraries.
The reported overlap between OpenAI's and Google's use of YouTube-derived transcripts increases the value of differentiated data sources and of rights, access, and processing capabilities around them.
Third-order effects
If high-quality text becomes the limiting input, frontier AI competition may increasingly be shaped by control of proprietary content ecosystems and the ability to transform non-text media into usable training data.
The pattern points toward a more commercialized training-data market, though the durability of that shift depends on whether new data sources and training approaches can relieve the reported constraint.
The trend: Frontier-model development is turning training data from a broadly available web resource into a strategic input governed by platform access, processing capacity, and content ownership.
Complete and utter waffle. There's exabytes of data. The problem is there isn't gigabytes of labeled data. What do humans think is interesting about the data, and have they tagged it so the computer can focus on this? The data needs to be clustered, classified, and extracted …
I don't know why this keeps repeating.. this linear math and assumption that the amount of data we have recorded digitally is fixed somehow. Some of the machines that we're building just create data extremely large amounts.. it's highly organized data based on the natural world …
Taking a quick break from a family trip to share my latest: it's generally understood that more data = better AI models. But what happens when you've exhausted the internet? We're rapidly getting to that point of data drought. …
AI companies are running out of high quality data to train the next LLMs. “Harnessing all the high-quality language and image data available could still leave a shortfall of 10 trillion to 20 trillion tokens or more...” https://www.wsj.com/...
This is throwing another “s” word against the wall and hoping it will stick. There is only one path forward and that is proper licensing of authoritative content and data. #MediaRevenue
A limiting factor of LLM-based artificial intelligence is training data. There's a real challenge that there simply isn't enough openly available data in certain fields to effectively train a generative AI. Generating games using LLMs is a simple example of where it's a problem. …
As journalism retreats from AI behind walls, the real question we need to address is what is missing from the web: the voices of people who did not have the power to publish in history... For Data-Guzzling AI Companies, the Internet Is Too Small https://www.wsj.com/...