GB300 NVL72: Anatomy of a 120 kW Rack-Scale AI Factory

Overview

The NVIDIA GB300 NVL72 (Blackwell Ultra) marks the point where the rack, not the GPU, becomes the unit of compute. Seventy-two Blackwell Ultra (B300) GPUs and 36 Grace CPUs operate as a single NVLink domain delivering 1.08 ExaFLOPS of dense FP4, 20 TB of unified HBM3e memory, and roughly 50x the AI-factory output of a Hopper-generation system. A sustained 120 kW power draw and a 1.36-tonne footprint make direct-to-chip liquid cooling mandatory rather than optional. This reference architecture summarises the silicon, the rack, the cooling loop, the deployment topology, and the inference economics behind the investment.

What you will learn

• The Blackwell Ultra (B300) step-up: 15 PFLOPS FP4, 288 GB HBM3e at 8 TB/s, 1,460 W TDP.
• How 72 GPUs plus 36 Grace CPUs act as one NVLink domain (130 TB/s, 20 TB unified memory).
• Why a 1,400 W-per-chip heat load forces a direct-to-chip liquid loop with an in-rack CDU.
• A 16-week, power-plant-grade deployment blueprint spanning power, cooling, and software.
• The economics: up to 25x tokens/sec versus Hopper and up to 35x lower cost per token.

The silicon: Blackwell Ultra (B300)

• 15 PFLOPS FP4 dense compute, a 66.7% lift over B200 for MoE inference.
• 288 GB HBM3e per GPU running at 8 TB/s.
• 1,460 W TDP, a 40% higher thermal limit that sustains boost clocks through heavy GEMM phases.
• Faster softmax/attention path that removes the Hopper and B200 attention bottleneck.

The rack: 72 GPUs as one NVLink domain

Eighteen compute trays house 72 Blackwell Ultra GPUs and 36 Grace CPUs (Arm Neoverse V2). Nine 5th-generation NVLink switch trays deliver 130 TB/s of aggregate bandwidth, with measured multi-node NCCL bus bandwidth near 836 GB/s. A unified memory fabric fuses 20 TB of GPU HBM3e with 17 TB of Grace LPDDR5X into one coherent space, removing cross-rack sharding latency for trillion-parameter models.

Generational step-function

Spec	H100 node	B200 NVL72	GB300 NVL72
FP4 dense	N/A	720 PFLOPS	1,400 PFLOPS
Total memory	~1.1 TB	13.4 TB	20 TB
Power / rack	~10 kW	~120 kW	~120 kW
Cooling	Air / RDHx	Direct liquid	Direct liquid

Cooling: the direct-to-chip liquid loop

A 1,400 W heat load per chip is captured by copper cold plates with 0.15-0.2 mm micro-channels targeting a low thermal resistance. Coolant flows at 20 L/min per GPU through 252 blind-mate quick connects and flow-balancing manifolds (delivery uniform within +/-5%). The rack strictly requires an in-rack Coolant Distribution Unit sized for 150-200 kW.

Deployment: a 16-week blueprint

Weeks 1-3, foundation and load study (floor loading, cooling-water spec, electrical capacity). Weeks 4-6, power (480 V three-phase switchgear, CDU placement, manifold pipe runs). Weeks 7-9, mechanical integration (1.36-tonne cabinet anchoring, coolant fill, pressure and leak testing). Weeks 10-16, network and software (InfiniBand/ConnectX-8 fabric, Triton/MLOps stack, plant-LLM production launch). The topology bridges OT and AI zones: plant floor, hardened data-aggregation bridge, the isolated AI infrastructure room, and the operations control room.

The economics: inference dominance

By 2026, inference is projected to take 55% of AI-optimised infrastructure spend, overtaking training, while inference cost falls roughly 10x annually. The GB300 NVL72 generation cuts per-token cost up to 35x versus Hopper, which reframes AI infrastructure as an operating-cost decision, not just a capability one.

Deploy now, or wait for Rubin?

For most operators the recommended path is to deploy B200/GB300 now and capture immediate inference ROI, avoiding the cost of idle or delayed compute; current-generation pricing has historically held up across next-generation announcements. Waiting for Rubin is warranted mainly for teams training trillion-parameter frontier models or needing million-token-context acceleration. The throughline: deploy Blackwell today, and prepare the facility (480 V power, liquid-cooling loop) for Rubin tomorrow.

Frequently asked questions

Download the full blueprint (PDF · 15 pages)

Download the full blueprint (PDF)