Intel debuts Gaudi 3, set for Q3 mass production, claiming up to 1.7x the training performance, 50% more inference, and 40% better efficiency than Nvidia's H100

Tom's Hardware 2024-04-10 Paul Alcorn

Context & Ripple Effects

Gaudi 3 moves Intel’s AI-accelerator effort from an earlier announcement with limited technical detail to a stated production timetable and explicit comparisons with Nvidia’s H100. Intel had positioned the chip as its answer to both Nvidia and AMD’s new accelerator line, after first outlining Gaudi 3’s 2024 launch.

The comparison arrives as AMD had also claimed a performance advantage over H100 for MI300X inference workloads, making benchmark claims and availability central inputs to buyers evaluating alternatives to Nvidia. Intel’s own leadership had argued that inference would become the more important AI workload, which makes Gaudi 3’s inference claim particularly consequential to its strategy.

First-order effects

Intel gains a concrete Q3 production target and a performance-and-efficiency pitch for prospective AI infrastructure customers; Nvidia’s H100 becomes the immediate comparison point for Gaudi 3 evaluations.
Buyers considering new accelerators can assess a third major supplier against H100 on training, inference and efficiency claims, although real-world workload validation will determine their purchasing value.

Second-order effects

AMD and Nvidia face added pressure to defend performance, availability and software support rather than relying on accelerator specifications alone; AMD had already positioned MI300X against H100 in inference.
Procurement teams may broaden pilots across Nvidia, AMD and Intel, increasing the importance of total system cost, deployment tooling and workload-specific benchmarks alongside chip-level performance.

Third-order effects

If alternative accelerators reach volume and meet their claims, AI compute purchasing could become more heterogeneous, weakening the degree to which a single accelerator baseline defines infrastructure choices.
The durable competitive test shifts toward integrated hardware, software and customer deployment support: benchmark leadership alone may not translate into share without an ecosystem customers can operate at scale.

The trend: AI accelerator competition is moving from announced roadmaps to production readiness and workload-specific procurement among multiple suppliers.

Discussion

@dylanonchips @dylanonchips on x
Emphasizing that @Dell is “working hard to meet the demand and make AI adoption really easy for these enterprise customers,” the company's founder finished his brief talk with a plea for @intel CEO Pat Gelsinger. “What we need is more Gaudi 3 in volume.” https://www.crn.com/...
@iancutress @iancutress on x
The person (I'm under the impression) that pushed through Xeon Scalable was formerly at Staples and Gillette. I always wondered what about product naming razors and stationery made you an expert in semiconductor enterprise marketing. He now works at AMD. [image]
@intelnews @intelnews on x
Using large language model-based generative AI solutions to enhance productivity. Learn about an on-prem, secure, #IntelXeon and Gaudi-powered solution built in collaboration with Articul8, a partner company, and how Intel has already been using it in the fabs. #IntelVision [imag…
@intelnews @intelnews on x
Intel is providing you the flexibility to deploy the AI system YOU need in your enterprise. The AI NIC is a network interface card, providing optimized Ethernet-based network connectivity for training and inference on the largest AI clusters. https://www.intel.com/... [image]
@never_released @never_released on x
Alright, the Gaudi3 whitepaper doesn't seem to be in an ideal shape. An overlapping table somewhere, a page with only two words, and... spot the accidental swastika too
@tobias_writes Tobias Mann on x
I'm not sure why Intel was so keen to talk FP8 FLOPS for Gaudi3 The more impressive spec IMHO is BF16/FP16 perf. #Intel blows #Nvidia and #AMD out of the water at half precision. You can find my full two-page write up @theregister here: ⬇️ https://www.theregister.com/ ... [image]
@mattkimball_mis Matthew Kimball on x
OK - it's official, @PGelsinger is dancing on stage with Gaudi 3. Introducing @intel's latest AI accelerator -across the board incredible numbers related to perf per watt and perf per dollar - training and inference. Available as a mezz card, PCIe card or a baseboard.... [image]
@cnbcovertime @cnbcovertime on x
“Overall, I don't think Intel is getting as much credit as they deserve, primarily for what their foundry business will be bringing in,” says @PatrickMoorhead on $INTC's new Gaudi 3 AI chip announcement. “It gets much more interesting when they bring an AI GPU to market in 2025.”…
@intelnews @intelnews on x
Granite Rapids will FUNDAMENTALLY change the way AI will be implemented in the enterprise. ⭐ On stage at #IntelVision, @PGelsinger ran a 70 billion parameter 4-bit model entirely on Xeon in just 86 milliseconds. https://www.intel.com/... [image]
@danielnewmanuv Daniel Newman on x
The TLDR is that it isn't a zero sum game . @intel Gaudi 3 (and 2 for that matter) is a compelling part but there is a lot of software and ecosystem work to gain adoption and make it “market competitive” with @Nvidia. The claims on efficiency and speed are definitely noteworthy..…
@iancutress @iancutress on x
More silicon news: @Intel lifted the lid on Gaudi 3 today at #IntelVision. OAM and PCIe versions. ➡️TSMC N5 ➡️PCIe 600W ➡️OAM 900W air, 900W+ liquid ➡️128 GB HBM2e ➡️64 Tensor Cores ➡️24x200 GbE ➡️PCIe 5.0 x16 ➡️Supports clusters up to 8192 ➡️2xFP8, 4xBF16 vs Gaudi 2 ➡️10 tiles […
@tobias_writes Tobias Mann on x
@IanCutress @intel Official comment from Habana: “Our methodology was to use only IPs that were already proven in silicon before we tape out. At the time we taped out Gaudi3 there was simply no available physical layers that were validated to meet our standards,” COO Eitan Medina…
@danielnewmanuv Daniel Newman on x
The potential impact of Gaudi3 on nvidia? The performance is probably less critical than its developer ecosystem and the software abstractions that make Gaudi 3 accessible for AI workloads. Over time, as layers of abstraction move up the stack and Intel delivers on the promise of…
@intel @intel on x
Together, we are making AI accessible for enterprises at the edge. By leveraging Retrieval Augmentation, we empower organizations to develop unique solutions, propelling businesses into an AI-driven future. Learn more ⬇️ #IntelVision
@iancutress @iancutress on x
Whitepaper is now live. It's certainly more detailed than previous Gaudi disclosures! I'm sending questions back to the Intel team, mostly clarifications on exact numbers etc
@aschilling Andreas Schilling on x
How Intel plans to scale out #Gaudi3 for up to 8,192 accelerators with its Ethernet capabilities. [image]
@intel @intel on x
🎤We've got some of the brightest minds in business and tech at #IntelVision 2024 ready to share insights on how AI is powering business optimization. We're bringing #AI everywhere with @intel #silicon, #software, and services. Watch here: https://www.linkedin.com/... #Intel [imag…
@intelnews @intelnews on x
📞Calling all global enterprises looking to deploy GenAI. Intel's Gaudi 3 AI accelerator delivers 50% better inference and 40% better power efficiency than Nvidia H100 - at a fraction of the cost. [image]
@intelai @intelai on x
The #IntelGaudi 3 #AI accelerator offers a highly competitive alternative to NVIDIA's H100 with higher performance, increased scalability, and PyTorch integration. Explore more key product benefits. https://www.intel.com/... [video]
@dylanonchips @dylanonchips on x
.@intel said its upcoming Gaudi 3 AI accelerator chip can best Nvidia's powerful H100 GPU for training large language models and offer similar or, in some cases, better performance than the rival's memory-rich H200 for large language model inferencing. https://www.crn.com/...
@emostaque @emostaque on x
Tbh I got lots of flack for backing the Intel chips on a technical basis but good chips are good chips and they have done a huge amount of work on open XLa, sycl-CUDA conversion and building out openVINO etc. definitely check out the dev cloud and build stuff for NPUs too [image]
@bentheegg Benjamin Lefaudeux on x
Gaudi3 out, looks like a much beefier version of Gaudi2 + emphasis on interconnect, a bit like TPUs. Quoted numbers are fp8 which is non trivial at the moment, but supposedly pytorch compatible (native pytorch ops or intermediate like ONNX ?) https://www.nextplatform.com/ ...
@intel @intel on x
Announced at #IntelVision, Gaudi 3 offers an open-source AI software stack for better #GenerativeAI performance and is projected to average 1.5x faster time-to-train. Learn more ⬇️ #LLM #Intel #IntelGaudi
@iancutress @iancutress on x
At #IntelVision, it was announced that Intel is dropping the Xeon Scalable Processor branding going forward. 6th gen will simply be called Xeon 6. It's a good change. Warning, it's not retroactive (like Wi-Fi was). SPR is still 4th Gen Xeon SP, EMR is still 5th Gen Xeon SP. [imag…
@intelnews @intelnews on x
Developing open, secure GenAI systems that deliver best-in-class ease of deployment, performance and value enabled by retrieval-augmented. @Intel, @SAP, @RedHat, @VMware and others are committed to developing an open platform for enterprise AI. https://www.intel.com/... [image]
@marcochiappetta Marco Chiappetta on x
. @intel just took the wraps off its new #Gaudi3 #AI accelerator and announced the #Xeon6 line-up, comprised of both Sierra Forest and Granite Rapids-based chips, at #Vision2024. Some highlights and thoughts available over on @Forbes for those interested. https://www.forbes.com/.…
@kristinaparts Kristina Partsinevelos on x
I just asked Intel's CEO @PGelsinger about Gaudi 3 cost, he said “it's not a little below, but a lot below” Nvidia's offering. Much like Nvidia, Intel not giving exact pricing today but shows how they plan to compete. $INTC $NVDA
@intelnews @intelnews on x
At #IntelVision, customers shared their successes with Gaudi accelerators and their commitment to Intel's enterprise AI approach. Read the thread to learn about improved productivity, simplified deployments, and accelerated revenue generation from Intel's solutions. [image]
@intelnews @intelnews on x
Open, secure, turnkey solutions to deploy GenAI using your own data make the difference between successful deployments and AI failures. Intel is working with the ecosystem to provide a “secure easy button” for enterprise AI. Learn more. https://www.intel.com/... [image]
@intelnews @intelnews on x
“We are excited to be adding Gaudi 3 AI Accelerators to our PowerEdge XE9680 lineup this year! It's a game-changer. Gaudi 3 strengthens our partnership by addressing customer infrastructure needs, reducing TCO and easing deployment.” -@MichaelDell #IntelVision [image]
@kristinaparts Kristina Partsinevelos on x
Intel says its new Gaudi 3 chip is over 2x as power-efficient as and can run AI models 1.5x faster than Nvidia's H100 GPU. $INTC $NVDA Details here:
@emostaque @emostaque on x
Told you so, Gaudi3 also way lower TCO, particularly with the Ethernet interconnect (3.7 Tb/s) vs custom which ramps pricing for pods. Will be pretty easily available too, lots of diffusion transformers everywhere... Good job team @HabanaLabs 👏 [image]
@beth_kindig Beth Kindig on x
The Gaudi 3 $INTC will start shipping later in 2024, and Intel says it has already received commitments from Dell $DELL, Super Micro $SMCI, HP Enterprise $HPE, and Lenovo to build Gaudi 3 systems and servers.
@dellservers @dellservers on x
🌐 The Dell #PowerEdge XE9680 takes a giant leap in #AI & HPC with @intel Gaudi 3 AI Accelerator! 🚀 Dive into our blog to see how it's reshaping the future of #GenAI acceleration, offering unparalleled support for complex workloads: https://www.dell.com/...
@intelnews @intelnews on x
The #IntelVision keynote with CEO @PGelsinger has kicked off! Follow along as Intel unleashes enterprise AI with Gaudi 3, AI open systems strategy and new customer wins: https://www.intel.com/... [image]
@aschilling Andreas Schilling on x
The Gaudi 3 AI accelerator is composed of two compute dies, which are identical, while one is rotated for packaging and build the interconnect. The schematics by Intel are not correct, but simplified to show the I/O with 24x 200GbE, PCIe 5.0 x16 and Media Engine. [image]
@aschilling Andreas Schilling on x
This is the #Gaudi3 OAM HL-325L - 64 Tensor Cores (5th Gen) - 128 GB HBM2 - 96 MB SRAM - 24x 200GbE - PCIe 5 x16 - 900 W (air cooled) - +900 W with water - two compute dies - 5 nm TSMC https://www.hardwareluxx.de/ ... [image]
@_fabknowledge_ @_fabknowledge_ on x
*INTEL UNVEILS NEW AI ACCELERATOR PROCESSOR CALLED GAUDI 3 *INTEL SAYS NEW GAUDI 3 PRODUCT OUTPERFORMS NVIDIA'S H100 CHIP nice bit of news needed
r/technews r on reddit
Intel reveals details of new AI chip to fight Nvidia dominance
r/intel r on reddit
Intel Introduces Gaudi 3 AI Accelerator: Going Bigger and Aiming Higher In AI Market

Chronicles