Looking back at “Deep Learning is Hitting a Wall”, a 2022 article by Gary Marcus, ridiculed by many, and how a paradigm shift, not just scale, is needed for AGI
His predictions are very accurate. [embedded post] X: Gary Marcus / @garymarcus : @ocs568BD i predicted hallucinations in 2001; here is another list from two years ago today Gary Marcus / @garymarcus : Two years ago today I argued that Deep Learning was Hitting a Wall. How did I do? Point by point evaluation at link below [image] Elliot Murphy / @elliotmurphy91 : Deep learning is (still) hitting a wall [image] LinkedIn: Eduardo César Garrido Merchán : Hallucinations are not such, mainly because LLMs are not formal systems. Everything that they do is pure syntax, they do not verify the information. … Forums: Hacker News : Two years later, deep learning is still faced with the same basic challenges
Context & Ripple Effects
This is a renewed case for the critique first outlined in coverage of deep learning’s limitations and neurosymbolic alternatives. It lands amid a split in related coverage: one view treats flawed current LLMs as early AGI, while another warns against building on assumptions that generative AI will be transformative.
Marcus’s retrospective does not establish that scaling has failed; it sharpens the dispute over whether persistent hallucinations and weak verification are engineering problems or evidence that a different architecture is needed.
First-order effects
- The article gives Gary Marcus and other paradigm-shift advocates a fresh reference point in the AGI debate, centered on his claimed record forecasting hallucinations and other limitations.
- It puts more scrutiny on the proposition that larger deep-learning systems alone can deliver reliable, general intelligence, rather than treating model scale as the decisive path.
Second-order effects
- Labs advancing scale-first roadmaps face added pressure to show that reliability and verification improve materially, not merely that models perform better on broad benchmarks.
- The piece strengthens the standing of research agendas that combine statistical models with explicit reasoning or verification, echoing the earlier case for neurosymbolic approaches.
Third-order effects
- If the same failure modes remain central as models scale, AGI competition could shift from primarily accumulating compute and data toward differentiating architectures that can reason and verify outputs.
- The underlying divide may persist: the view that today’s LLMs are nascent AGI can coexist with the claim that they require a substantive paradigm change before they become dependable general systems.
The trend: The story is one data point in the widening contest between scale-first AI development and approaches that treat reliability, reasoning, and verification as architectural requirements.