ChatGPT testing shows the chatbot struggles with basic arithmetic questions, confidently offering entertaining but wrong answers, an inherent problem with LLMs
Josh Zumbrun / Wall Street Journal :
Context & Ripple Effects
Josh Zumbrun's Wall Street Journal testing found ChatGPT answering basic arithmetic questions with fluent, entertaining, and wrong answers — not a bug but a structural property of a model that predicts plausible text rather than computing. The finding landed in early February 2023, when ChatGPT was being pitched as a general-purpose answer engine.
Subsequent coverage kept confirming the pattern rather than refuting it: Khan Academy's ChatGPT-powered tutor Khanmigo was still making basic arithmetic errors a year later, an analysis of 517 programming answers found 52% contained incorrect information, and researchers documented cases where GPT-3.5 outperformed GPT-4 at math as models drifted between versions.
First-order effects
- Users treating ChatGPT as a calculator or factual reference get wrong answers delivered with full confidence, and OpenAI's positioning as a reliable answer engine takes the direct hit.
- Every downstream product built on the same models inherits the flaw immediately — no wrapper or prompt fixes what the underlying architecture cannot compute.
Second-order effects
- Education deployments feel it hardest: Khan Academy has to build error-correction workarounds into Khanmigo because a tutoring bot that flunks arithmetic undermines its core promise.
- Confident wrongness shifts buyer attention toward verification layers and domain-specific tools, since the Stack Overflow analysis shows verbosity amplifies the problem by dressing misinformation in authoritative prose.
Third-order effects
- If fluency and correctness stay decoupled, LLMs get pushed toward hybrid designs — calling external calculators, code execution, or retrieval — rather than raw generation, reshaping what 'an AI answer' even means.
- The drift findings suggest version upgrades do not monotonically improve accuracy, so enterprises building on these APIs need regression testing per model release as standard practice.
The trend: Large language models are being forced from confident free-form generation toward verified, tool-augmented answers as their arithmetic and factual failure modes become a deployment blocker.