An in-depth look at Nvidia's DGX H100 setup: 32 DGX boxes, each weighing ~300lbs and housing eight individual $25K H100 GPUs, a cooling system, and other chips
Built to drive the graphics of videogames including ‘Call of Duty, ’ they now also power ChatGPT and other AI tools
Context & Ripple Effects
Nvidia’s DGX H100 is part of an expanding line of packaged AI systems: the company had already positioned the DGX GH200 as a supercomputer platform, building on earlier multi-GPU AI and high-performance-computing designs such as HGX-2.
The setup makes the infrastructure behind AI workloads tangible: a deployment spans 32 heavy, cooled boxes and 256 H100 GPUs, rather than a collection of standalone accelerators. That systems-level packaging matters as generative AI turns GPU access into an operational data-center question.
First-order effects
- A full DGX H100 deployment concentrates 256 H100 GPUs—about $6.4 million in GPUs at the stated per-unit price—into a single Nvidia-supplied configuration, alongside cooling and supporting chips.
- Buyers running AI tools such as ChatGPT must provision for the physical footprint, cooling, and integration of 32 DGX boxes, not merely acquire accelerator capacity.
Second-order effects
- Nvidia captures more of the deployment value by selling an integrated system; customers have less latitude to treat GPUs as interchangeable components once compute, cooling, and supporting hardware are bundled.
- Data-center operators and enterprise AI teams face infrastructure planning as a gating factor for AI capacity, reinforcing the advantage of vendors that can deliver validated compute systems rather than chips alone.
Third-order effects
- If packaged deployments keep scaling, AI competition will increasingly hinge on the ability to finance, install, cool, and operate dense compute fleets—not only on model development.
- The pattern points toward a more vertically integrated AI-infrastructure market, where accelerator suppliers’ system, networking, and deployment capabilities can shape buyer lock-in and capacity access.
The trend: AI compute is shifting from discrete GPU procurement toward integrated, facility-aware systems built to operate dense accelerator clusters.