Benchmarking AMD's MI300X and Nvidia's H100 and H200; in theory, AMD's GPU has advantages in specs and total cost of ownership, but software bugs hold it back
This NVidia monopoly on #ai hardware is not good for anybody. … X: Nicholas Wilt / @cudahandbook : NVIDIA to churn the hardware instruction set with abandon, sometimes even committing featurecide and relying on the PTX translator to emulate instructions that were removed. The PTX translation code is in the driver as well as the offfline toolchain (ptxas) and is 3/x Nicholas Wilt / @cudahandbook : supercomputers in the world. So yeah, CUDA is a deep, deep moat. I get pretty offended when folks intimate that any luck was involved. We knew exactly what we were doing and why. /fin Nicholas Wilt / @cudahandbook : why I was making sure it ran on Windows as well as Linux. It was healthy for the code base. Today, NVIDIA has parlayed CUDA's Windowa support into a monopoly position in GPU workstations, because 1,200 workstation apps use CUDA. Another pillar is PTX, which enables 2/x Nicholas Wilt / @cudahandbook : multithreaded, so it can exploit modern multicore CPUs for performance gains proportional to the core count. Another triumph of software engineering. All of this great software runs on a span of platforms from tiny SOCs for cars and drones and robots, to the biggest 4/x Nicholas Wilt / @cudahandbook : CUDA's software stack has a few distinct pillars that are triumphs of software engineering (let alone software architecture). The driver API was built in C, portable across both operating systems and CPU architectures. Across the 6 years I worked on CUDA, no one questioned 1/x George Hotz / @realgeorgehotz : “AMD's software experience is riddled with bugs rendering out of the box training with AMD is impossible.” “It's not just that it's immature software, they need to change how they do development.” Good luck @dylan522p! You Jiacheng / @youjiacheng : > Tensorwave, the largest AMD GPU Cloud has given GPU time for free to a team at AMD to fix software issues, which is insane given they paid for the GPUs. This is INSANE... Indrajit Bhosale / @devnull_0 : Recently tried to (painstakingly) write a convolution layer kernel using AWS Trainum with their NKI programming model (their latest and greatest), only to realize CUDA moat is well and truly alive! Dylan Patel / @dylan522p : Our 5-month journey conducting independent analysis & benchmarking of AMD MI300X vs Nvidia H100 + H200 Detailed, open source low-level benchmarks performance vs TCO Comprehensive public recommendations It's not just immature software, they need to change how they do development @vintrotweets : > Tensorwave, the largest AMD GPU Cloud has given GPU time for free to a team at AMD to fix software issues, which is insane given they paid for the GPUs. glad we got the tinybox green Ben Thompson / @benthompson : This is incredible work. And yeah, AMD has sucked at software for 50 years. Ben Bajarin / @benbajarin : Repeat after me the reason Nvidia GPUs remain so well positioned - massive (and growing installed base) and architectural compatibility. Good analysis below. Saurabh Dash / @theycallmemr_ : “AMD” is insane because it appears to be one of the most legitimately dangerous companies with the potential to gigafry the market but exclusively employs literal turbonormies who unironically want to like design x86 processors and basically get oneshotted by their own drivers. Daniel Lemire / @lemire : NVIDIA's GPUs excel due to their mature CUDA software stack, which provides better performance in practical scenarios even when they have inferior hardware specs. via @GeimanThiesen [image] LinkedIn: Phillip Hendrickson, Ph.D. : Commentary about SemiAnalysis' MI300X deep dive is currently making the rounds in the news and on social media platforms, understandably so given what's in the article. …
Context & Ripple Effects
This benchmark sharpens a long-running tension in AI compute: earlier coverage described a gradual erosion of CUDA's monopoly in ML development, but this comparison shows why a mature software ecosystem can still outweigh favorable hardware specifications.
It also follows coverage of H100 supply constraints, where access to Nvidia hardware was already a strategic issue. The MI300X gives buyers a potentially lower-TCO alternative on paper; the reported bugs and tooling gaps determine whether that alternative is usable in production.
First-order effects
- AMD and MI300X users cannot reliably capture the chip's stated specification and total-cost advantages until driver and toolchain issues are resolved; Tensorwave's provision of GPU time directly supports that remediation effort.
- Nvidia's H100 and H200 retain a practical performance advantage for workloads dependent on the mature CUDA and PTX stack, despite the benchmark's hardware-level comparison.
Second-order effects
- AI infrastructure buyers must evaluate deployed software compatibility and debugging burden alongside chip price and raw throughput, raising the switching cost from Nvidia platforms.
- AMD's cloud and software partners face pressure to contribute testing capacity, fixes, and developer support; otherwise, prospective MI300X demand may not translate into production deployments.
Third-order effects
- The comparison reinforces that AI accelerator competition is increasingly decided by the integrated hardware, drivers, compilers, and framework stack—not by chip specifications alone.
- If AMD's software maturation continues, independently reproducible benchmarks could make TCO a more consequential basis for multi-vendor GPU procurement; until then, CUDA remains a durable source of market power.
The trend: AI accelerator competition is shifting from a race over peak specifications toward a contest over complete, production-ready software ecosystems and verifiable cost performance.