Nvidia unveils the GH200 Grace Hopper Superchip, a combination GPU and CPU relying on high-bandwidth memory 3, or HBM3e, expected to enter production in Q2 2024
BloombergIan King
Context & Ripple Effects
This is the productization step after Nvidia’s earlier Grace CPU plan: pairing its Arm-based server CPU with a GPU turns Grace from a standalone processor strategy into a tightly coupled data-center offering.
The emphasis on HBM3e also foreshadows the later H200 memory upgrade and the subsequent GB200 CPU-GPU pairing, making memory bandwidth and system integration central to Nvidia’s accelerator roadmap.
First-order effects
Nvidia gives data-center customers a defined Q2 2024 production target for a combined Grace CPU and Hopper GPU platform, rather than requiring those components to be evaluated as separate products.
The design makes HBM3e a direct part of the platform’s value proposition, placing memory bandwidth alongside compute as a key purchasing criterion.
Second-order effects
Server buyers and Nvidia’s ecosystem must plan deployments around a more integrated CPU-GPU architecture, potentially narrowing the role for separately selected host CPUs in workloads suited to the platform.
Demand shifts toward the high-bandwidth-memory supply chain and competing accelerator vendors face pressure to match both memory capability and CPU-GPU integration.
Third-order effects
If this product pattern persists, AI infrastructure competition will increasingly be decided at the system level—accelerator, host CPU, memory, and interconnect together—rather than by GPU specifications alone.
The roadmap points to the memory wall becoming a durable constraint on AI compute: advances in accelerator performance will depend increasingly on access to and integration of high-bandwidth memory.
The trend: Nvidia is moving toward heterogeneous AI systems in which tightly integrated compute and memory, not standalone chips, define the platform.
NVIDIA's architecture code names are catching up to our times, from Kepler (1571-1630) now to Grace Hopper (1906-1992) - one of the first programmers, working on the Harvard Mark I during World War II.
Nvidia $NVDA just announced a new AI chip configuration, the Grace Hopper Superchip (GH200), which tied together Nvidia's H100 chip with an Nvidia central processor. The GH200, expected to speed up generative AI applications like ChatGPT $MSFT, is expected to be available in Q2..…
.@nvidia 's new module pairs a 72-core ARM Neoverse V2 CPU with their H100 GPU on a single module CPU and GPU are connected with 900GB/s NVLink interconnect. 7X faster than the normal PCIe Gen5 x16 link. This close integration has more benefits beyond raw bandwidth [image]
🚦 $NVDA Unveils Next-Generation GH200 Grace Hopper Superchip Platform for Era of Accelerated Computing Generative AIWorld's First HBM3e Processor Offers Groundbreaking Memory, Bandwidth; Ability to Connect Multiple GPUs for Exceptional Performance; Easily Scalable Server Design
An additional NVLink connection allows pairing up with another GH200 CPU+GPU combo, giving even further easy access to memory on peer modules in the same way [image]
With a normal x86 CPU attached to a GPU via PCIe, the GPU and CPU have separate memory page tables (1st picture), requiring extra steps to share memory. In the GH200, the CPU and GPU, processes share a combined page table between CPU and GPU memory (2nd picture) [image]
Why are people saying Jensen is announcing the GH200? I thought that's what Computex was. I've had a tab open with the whitepaper with the tech details ever since.
.@nvidia CEO Jensen Huang announces the GH200, a 72-Core CPU combined with a 4 PFlop Hopper GPU with a whopping 141 GB of HBM3e, 5TB/s of memory bandwidth #SIGGRAPH2023 [image]