OpenAI researchers argue that language models hallucinate because standard training and evaluation procedures reward guessing over admitting uncertainty
Read the paper(opens in a new window) — At OpenAI, we're working hard to make AI systems more useful and reliable.
OpenAI
Context & Ripple Effects
OpenAI’s earlier work on process supervision had already framed reliability as a training-design problem rather than simply a matter of producing more correct final answers. The company later reported that o3 and o4-mini hallucinated more often than prior models, making the evaluation incentives behind model behavior an immediate operational concern.
This paper sharpens that arc: if benchmarks score an unsupported answer more favorably than an explicit “I don’t know,” apparent capability and calibrated reliability can diverge.
First-order effects
OpenAI’s training and evaluation teams gain a concrete target for reliability work: reward calibrated uncertainty and distinguish abstention from an incorrect guess.
Users and evaluators should treat high answer rates as an incomplete quality signal, particularly where a model lacks enough information to answer reliably.
Second-order effects
Model providers competing on benchmark results may need to revise scoring regimes that penalize abstention, or risk optimizing for confident but unsupported outputs.
Enterprise deployments will place more value on testing and controls that measure when a model should defer, alongside tests of whether it can produce a correct answer.
Third-order effects
If evaluation practices shift toward calibration, model comparisons may move from single accuracy-style scores toward task-specific measures of correctness, uncertainty, and safe deferral.
The broader reliability stack could increasingly combine model-level training with operational assurance, because no single benchmark can establish whether an AI system knows its limits in production.
The trend: AI reliability is becoming an incentives-and-evaluation problem: developers are being pushed to reward calibrated uncertainty, not just answer completion.
IMO - this paper is right about a major contributing factor to hallucinations but wrong about the cause LLM hallucinations are closer to a *cache miss*. Well... not quite a single cache miss. Imagine needing to fetch from 50 caches to get it right... and if you miss 15
Paper from OpenAI says hallucinations are less a problem with LLMs themselves & more an issue with training on tests that only reward right answers. That encourages guessing rather than saying “I don't know” If this is true, there is a straightforward path for more reliable AI. […
Researchers from @OpenAI admitted that hallucinations of language models are caused, among other things, by the chosen training method and the evaluation of answers. Hallucinations stem from the structure of training. Models learn to guess the next token even if they are not [vid…
OpenAI realesed new paper. “Why language models hallucinate” Simple ans - LLMs hallucinate because training and evaluation reward guessing instead of admitting uncertainty. The paper puts this on a statistical footing with simple, test-like incentives that reward confident [image…
This OpenAI paper is actually pretty big - hallucination research typically focuses on fixing the models but the real problem seems to be more in how models are graded. The paper shows that models hallucinate bc benchmarks teach them to guess when uncertain instead of admitting
The new OpenAI paper “Why Language Models Hallucinate” is more like PR than research. The claim that hallucinations arise because training/evaluation reward guessing over abstaining is decades-old (reject option classifiers, selective prediction).
This OpenAI paper has some very compelling explanations on why hallucinations persist and what to do about it. Specifically in post-training, we should stop penalizing models for saying that they don't know an answer, and stop rewarding overly confident guessing. Eradicating [ima…
Open AI has a blog post about “why LLMs hallucinate” and (loosely) fess up that it's about the training objective. They show that gpt5 has an increased odds of abstaining from answer over o4-mini. — But this buries the punchline: — Users were unhappy with gpt5! It didn't li…
openai.com/index/why-la... so, contra the anti- consensus, hallucinations appear to be fixable, an artifact of simplistic post-training that doesn't properly account for uncertainty and reward abstention in evaluations.