An overview of the ML software development industry over the past decade: a decline of Nvidia's CUDA monopoly, PyTorch overtaking Google's TensorFlow, and more
the CUDA monopoly is nowhere close to being broken and CUDA will continue to be the key dependency for PyTorch. As a data point, Triton isn't the first rally — multiple vendors use the XLA compiler as a rally point. https://twitter.com/... @wholemarsblog : How Nvidia's CUDA Monopoly In Machine Learning Is Breaking - OpenAI Triton And PyTorch 2.0 https://www.semianalysis.com/ ... Calum Chace / @cccalum : The idea that ChatGPT spells doom for Google Search is almost certainly wrong. But Google's predominance in advanced AI is undermined by its failure to keep TensorFlow more popular that PyTorch. https://www.semianalysis.com/ ... Tren Griffin / @trengriffin : “Nvidia's FLOPS have increased multiple orders of magnitude by leveraging Moore's Law, but primarily architectural changes such as the tensor core and lower precision floating point formats. In contrast, memory has not followed the same path.” https://www.semianalysis.com/ ... @pommedeterre33 : An under appreciated fact (slightly exaggerated in article below): With current move to @openai Triton, PyTorch 2.0 makes Nvidia software moat a bit weaker. AMD & Intel are working to support Triton, easier than reinventing whole CUDA / Bias stack. https://www.semianalysis.com/ ... @jreuben1 : Breaking Nvidia's CUDA Monopoly In ML https://www.semianalysis.com/ ... - PyTorch 2.0 (Dynamic Shapes) - PrimTorch (~250 primitive operator building blocks) - TorchDynamo (partial / guarded graph capture + JIT recapture: ingest any PyTorch user script and generate an FX graph) Mathieu Orhan / @ai_unleashed : Hard to summarise this amazing post. 1) The memory-wall and how much it takes to correctly use a A100 2) PyTorch is winning and 2.0, along with OpenAI Triton, makes it realistic that actual competition with Nvidia emerges https://www.semianalysis.com/ ... @jreuben1 : - AOT Autograd - TorchInductor compiler (Wrapper Codegen) - OpenAI Triton (LLVM IR direct to PTX without cuBLAS / CudNN) - a good article ! @pommedeterre33 : This is the very reason why we bet on Triton instead of writing custom CUDA code for Kernl library, plus at the end when things do not work you review PTX instructions anyway... The main current advantage of CUDA is certainly maturity, but for how much time... https://twitter.com/...
Context & Ripple Effects
The framework war is effectively settled: back in 2019, an analysis of AI research papers already showed researchers abandoning TensorFlow for PyTorch while industry lagged behind — this piece confirms the crossover completed, with PyTorch now the dominant ML platform of the decade. But SemiAnalysis pushes back on the victory-lap narrative around OpenAI Triton, arguing that talk of a broken CUDA monopoly is wrong.
The counter-evidence is structural rather than rhetorical: Triton, introduced as an easier alternative to CUDA in OpenAI's 2021 launch and since backed by Meta, Microsoft, and Google per the FT coverage, integrates with PyTorch 2.0 yet still compiles down to Nvidia's stack — XLA was a previous rally point that never dislodged CUDA either.
First-order effects
- PyTorch's decade-long win over TensorFlow locks the ML ecosystem onto a framework whose key dependency remains Nvidia's CUDA, so every PyTorch deployment today still routes value to Nvidia regardless of which vendor's chip buyers think they are choosing.
- AMD and Intel are reported to be supporting OpenAI's Triton as their alternative ecosystem play, making Triton — not their silicon alone — the front line of their challenge to CUDA.
Second-order effects
- AMD's hardware case gets judged on software maturity: later benchmarking of the MI300X against Nvidia's H100 and H200 found spec and total-cost-of-ownership advantages eroded by software bugs, so the CUDA gap directly caps how much share AMD can convert from paper wins.
- Google's response runs through its own stack — the TPUv7 Ironwood generation positions Google as the most credible challenger to Nvidia precisely because it pairs custom silicon with a software path (XLA lineage) that does not depend on breaking CUDA at the framework layer.
Third-order effects
- If the Triton pattern holds — multiple vendors converging on one open kernel language above the driver — competition shifts from who owns the framework to who owns the compiler and runtime beneath it, keeping the chokepoint one layer down even as the visible API opens up.
- The decade's lesson for chip challengers is that hardware parity without a software ecosystem is not parity: expect future accelerator roadmaps to be priced and judged on software-stack completeness, with CUDA's entrenchment setting the bar every entrant must clear.
The trend: ML compute competition is consolidating around a software-stack battle — open frameworks like PyTorch on top, but compiler and driver layers like CUDA remaining the real moat that rivals such as AMD, Intel, and Google must route around.