At Intel's NYC event, Pat Gelsinger claimed that for AI, inference will become more important than training, and the industry wants to “eliminate” Nvidia's CUDA
Andrew E. Freedman / Tom's Hardware :
Context & Ripple Effects
Intel framed its AI challenge around the operational phase of AI systems and the software layer that helps preserve Nvidia's position. That framing became more concrete when Intel introduced Gaudi 3 as its competing AI accelerator, while later coverage described alternatives from Amazon, AMD and others as gaining particular relevance for inference.
The story is an early marker of a broader competitive argument: AI compute demand may not be decided solely by peak training performance, but by the cost, latency and software portability of serving models at scale.
First-order effects
- Intel publicly shifts its competitive message toward inference workloads, where buyers can prioritize deployment efficiency and latency alongside raw training throughput.
- CUDA becomes an explicit strategic barrier in Intel's argument, putting software portability and developer tooling at the center of competition with Nvidia.
Second-order effects
- Rival chip vendors and cloud providers have greater incentive to package inference-specific hardware, software and services; later coverage identified credible inference alternatives from Amazon, AMD and others.
- Customers evaluating AI infrastructure face a more consequential build-versus-lock-in trade-off: a CUDA-centric stack may simplify adoption, while alternative stacks must demonstrate workable migration paths and economics.
Third-order effects
- If inference continues to command a larger share of AI spending, the market could become more heterogeneous, with specialized accelerators competing alongside general-purpose training hardware rather than one architecture serving every workload.
- The durable competitive moat may shift toward software ecosystems, deployment tools and workload-specific performance—not only the ability to manufacture leading accelerators. Nvidia's later planned inference system involving Groq technology underscores that inference specialization can also be a response from the incumbent.
The trend: AI infrastructure competition is broadening from training-chip leadership into inference economics, latency and the software ecosystems that determine switching costs.