It is the most complete single-rack AI system NVIDIA ships in volume today, but "ultimate" needs three caveats. NVIDIA GB300 NVL72 puts 72 Blackwell Ultra GPUs and 36 Grace CPUs in one liquid-cooled rack. It has 37 TB of fast memory (about 20 TB of HBM3E plus 17 TB of LPDDR5X), 130 TB/s of aggregate NVLink bandwidth and 1.44 exaflops of sparse FP4 compute. That is enough to hold trillion-parameter models with room to spare and to serve them across a single NVLink domain. The caveats: the 37 TB is two tiers of memory, not one flat pool; frontier training still needs many racks; and NVIDIA's own Vera Rubin NVL72 is already being listed as its successor. For most organisations, the real question is not whether NVL72 is the ultimate AI factory but whether their workload needs a 72-GPU NVLink domain at all.
What is inside a GB300 NVL72?
| Component | GB300 NVL72 | Per GPU (calc) |
|---|---|---|
| Processors | 72 Blackwell Ultra GPUs, 36 Grace CPUs | One Grace for every two GPUs |
| GPU memory | About 20 TB HBM3E, up to 576 TB/s | 288 GB at 8 TB/s |
| CPU memory | 17 TB LPDDR5X, 14 TB/s | About 389 GB/s per Grace CPU |
| Fast memory total | 37 TB (NVIDIA figure) | — |
| NVLink | 130 TB/s aggregate through 18 NVSwitch chips | 1.8 TB/s bidirectional (900 GB/s per direction) |
| FP4 compute | 1.44 EF sparse / 1.08 EF dense | 20 PF sparse / 15 PF dense |
| Scale-out and offload | ConnectX-8 SuperNICs, BlueField-3 DPUs | 800 Gb/s (100 GB/s per direction) |
| Power and cooling | About 132–142 kW per rack (OEM figures); compute trays liquid-cooled | — |
Source: NVIDIA GB300 NVL72 page (accessed 6 Nov 2026); NVIDIA Inside Blackwell Ultra blog (22 Aug 2025); NVIDIA DGX GB300 SuperPOD reference architecture; Supermicro, Schneider Electric/Motivair and Vertiv rack power figures. NVIDIA publishes 1.2 MW per eight-rack scalable unit rather than a per-rack figure. Per-GPU columns are our arithmetic. Peak FLOPS are ratings, not measured throughput.
Is 37 TB really one pool of memory?
Not in the sense that every byte is equally fast. About 20 TB is HBM3E attached to the GPUs, at 8 TB/s per GPU. The other 17 TB is LPDDR5X attached to the Grace CPUs, at about 14 TB/s across the whole rack. That works out at roughly 389 GB/s per CPU, about a twentieth of one GPU's HBM bandwidth. NVLink-C2C makes the CPU memory coherent with the GPUs, so software can address it directly. In practice, model weights and active KV cache belong in HBM, and LPDDR5X works as a large, fast overflow tier for cache, embeddings and offloaded state.
Reaching another GPU's HBM is also slower than reaching your own: NVLink gives 900 GB/s per direction, against 8 TB/s locally, about a ninth. The rack behaves like one large memory system only for software that places data with care. We explained the same two-tier design on a single superchip in HBM3e vs LPDDR5X.
How large a model can it hold?
Weights take 2 bytes per parameter at FP16, 1 at FP8 and about 0.5 at FP4. Against roughly 20 TB of HBM:
- A 1-trillion-parameter model at FP8 needs about 1 TB for weights, about 5% of the rack's HBM. At FP4 it is about 0.5 TB.
- DeepSeek-R1 (671B parameters, about 689 GB as published) uses about 3% of the HBM. Llama 3.1 405B at FP16 (810 GB) uses about 4%.
- Full training state with mixed precision and the Adam optimiser is about 16 bytes per parameter. That puts about 1.3 trillion parameters in HBM before activations, or about 2.3 trillion if all 37 TB could be used.
So for inference, weights are not what fills the rack. What fills it is KV cache: long contexts and large batches of concurrent users can need far more memory than the weights. That is where 20 TB of HBM and 17 TB of LPDDR5X earn their keep. For training, the rack can hold the state of very large models, but holding a model is not the same as training it in reasonable time.
NVL72's memory is less about fitting one huge model than about keeping thousands of users' context resident, close to the GPUs, while 72 GPUs share the work over NVLink.
What does it change for distributed training?
The rack gives a 72-GPU NVLink domain for the communication-heavy parallelism patterns: tensor parallelism and expert parallelism. Microsoft's DeepSpeed team measured communication at 17% to 47% of iteration time on one to four DGX H100 nodes. The more of that traffic stays on NVLink, the less the GPUs wait.
But frontier-scale training still spans many racks. Meta trained Llama 3 405B on up to 16,384 H100 GPUs at 38–43% model FLOPs utilisation, and logged 466 job interruptions in a 54-day period. NVIDIA's GB300 SuperPOD treats eight NVL72 racks (576 GPUs, 1.2 MW) as one scalable unit, and its two-layer InfiniBand fabric reaches 16 units, or 9,216 GPUs. NVL72 is the building block, not the whole factory. See why NVIDIA uses both NVLink and InfiniBand.
What does it change for large-scale inference?
This is where NVIDIA makes its biggest claims. It projects "up to 50x" overall AI factory output for GB300 NVL72 against Hopper on a DeepSeek-R1 chart, and 10x user responsiveness. Both are vendor projections, labelled "subject to change". The more concrete mechanism is wide expert parallelism. Mixture-of-experts models send every token to its experts and back in every block, and a 72-GPU NVLink domain lets that all-to-all run at full speed. NVIDIA reports up to 1.8x more throughput per GPU with 32-way expert parallelism versus 8-way on DeepSeek-R1, a vendor claim. Treat these figures as direction rather than a forecast for your workload.
So is it the ultimate AI factory?
- Not the last word. NVIDIA's Vera Rubin NVL72 page says "Available Now". It lists 3 TB/s of NVLink 6 per GPU and 216 TB/s per rack, with HBM4 at 19.2 TB/s per GPU. Those figures were cut from the 3.6 TB/s, 260 TB/s and 22 TB/s announced in January 2026. AMD's Helios also offers 72 GPUs per rack.
- Not a drop-in. At 132–142 kW, one rack needs liquid cooling and a data hall built for it. Singapore's Green Data Centre Roadmap targets PUE of 1.3 or better. At CBRE's US$330–475 per kW per month colocation range, one rack's power works out at about US$44,000–67,000 a month.
- Not for every workload. Many inference fleets run models that fit on one 8-GPU server. Read NVL72 vs 72 individual GPUs before assuming you need the rack.
For how the 18 NVSwitch chips tie the rack together, see 72 GPUs, one AI supercomputer. For starting on one GB300 at the desk, see from DGX Station to GB300 NVL72.
Buying in Singapore
SourceIT does not list GB300 NVL72 racks. For data-centre projects, ask us to source through our partners; we will confirm availability, lead time and facility requirements rather than assume stock. The same Blackwell Ultra GPU is available on the desk in DGX Station GB300 systems, all price on request: the HP ZGX Fury G1n, MSI XpertStation WS300, Dell Pro Max with GB300, ASUS ExpertCenter Pro ET900N G3, GIGABYTE W775-V10-L01 and Supermicro ARS-511GD-NB-LCC. Each has 748 GB of coherent memory on one GPU, with no NVLink to other units. For rack servers, see the Lenovo ThinkSystem SR780a V3, price on request. For local model storage, the Micron 9550 PRO 7.68 TB Gen5 E3.S SSD is S$10,456.00 (listed 6 Nov 2026). Browse the AI Workstations and AI Servers collection. For a quotation, use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.
Frequently asked questions
Where does the 37 TB figure come from? NVIDIA counts about 20 TB of HBM3E on the 72 GPUs (288 GB each) and 17 TB of LPDDR5X on the 36 Grace CPUs as "fast memory". We found no NVIDIA page that uses "40 TB" for GB300 NVL72.
Is 1.44 exaflops the real speed? It is the peak sparse FP4 rating; dense FP4 is 1.08 exaflops. Real training runs reach a fraction of peak; Meta reported 38–43% model FLOPs utilisation for Llama 3 405B on H100.
Can one NVL72 train a frontier model? It can hold the training state of a model of over a trillion parameters, but frontier training runs use thousands of GPUs. NVIDIA's GB300 SuperPOD groups eight racks per scalable unit and scales to 9,216 GPUs.
Is the 50x claim against Hopper reliable? It is NVIDIA's projection on a DeepSeek-R1 chart, marked "subject to change". Use it as a direction, not a forecast, and benchmark your own model and latency targets.
How much power does a GB300 NVL72 rack need? OEMs such as Supermicro, Schneider Electric and Vertiv size it at about 132–142 kW per rack, with liquid cooling for the compute trays. NVIDIA states 1.2 MW for an eight-rack scalable unit.
