72 GPUs, One AI Supercomputer: Why NVIDIA GB300 NVL72 Needs NVLink, Not Just More GPUs
Because 72 GPUs only act as one machine if every GPU can reach every other GPU at close to its own memory-fabric speed. In NVIDIA GB300 NVL72, that job falls to NVLink and the NVLink Switch (NVSwitch). Each of the 72 Blackwell Ultra GPUs has 18 NVLink 5 links of 100 GB/s, bidirectional. Each link runs to a different one of 18 NVSwitch chips in the rack. So any GPU reaches any other in a single switch hop at 1.8 TB/s bidirectional (900 GB/s each way). Add up all 72 GPUs and you get the headline figure of 130 TB/s of aggregate NVLink bandwidth. Without this fabric, the same 72 GPUs would be nine or more separate servers talking over network cards with roughly a ninth of the bandwidth per GPU, and the largest models and parallelism schemes would slow down sharply.
How is the NVL72 NVLink domain built?
| Building block | GB300 NVL72 figure | What it means |
|---|---|---|
| GPUs and CPUs | 72 Blackwell Ultra GPUs, 36 Grace CPUs | Four GPUs per compute tray; each Grace pairs with two GPUs over NVLink-C2C |
| NVLink per GPU | 18 links x 100 GB/s = 1.8 TB/s bidirectional (900 GB/s per direction) | Per GPU, not per rack |
| NVLink Switch trays | 9 trays (1RU each), 2 NVSwitch chips per tray = 18 chips | One link from every GPU to every switch chip |
| Each NVSwitch chip | 72 ports, 7.2 TB/s all-to-all bidirectional | One port per GPU in the rack |
| Aggregate NVLink bandwidth | 72 x 1.8 TB/s = 129.6 TB/s, quoted as 130 TB/s | Sum of every GPU's injection bandwidth; no single transfer sees it |
| Half-rack to half-rack (calc) | 36 GPUs x 900 GB/s ≈ 32.4 TB/s per direction | Our inference from the single-hop design; NVIDIA publishes no bisection figure |
| In-switch compute | SHARP engines for in-network reductions and multicast | Part of an all-reduce can be done inside the switch |
| Scale-out per GPU | ConnectX-8 at 800 Gb/s = 100 GB/s per direction | NVLink gives each GPU 9x the off-GPU bandwidth of its network card |
Source: NVIDIA GB300 NVL72 page (accessed 6 Nov 2026); NVIDIA DGX GB rack user guide (updated 17 Sep 2026); NVIDIA NVL72 AI Factory reference architecture (updated 15 Sep 2026); NVIDIA HOT Interconnects 2025 NVLink slides (21 Aug 2025); NVIDIA Inside Blackwell Ultra blog (22 Aug 2025). The half-rack figure is our arithmetic.
Why can't you just add more GPUs over the network?
You can, and every large cluster does. But the network sets how the GPUs may cooperate. Training and serving a large model split the work in several ways, and each split has its own traffic pattern:
- Tensor parallelism splits each layer across GPUs. The partial results must be combined with an all-reduce at every layer, many times per token or training step. NVIDIA's own parallelism guidance keeps tensor parallelism inside the NVLink domain for this reason.
- Expert parallelism for mixture-of-experts models sends every token to the GPUs holding its experts and back again: an all-to-all exchange in every transformer block. DeepSeek reported that running this across nodes on H800 servers gave a computation-to-communication ratio of about 1:1, and dedicated 20 of each GPU's 132 SMs to communication.
- Pipeline and data parallelism are more tolerant. Pipeline stages pass activations point to point, and data parallelism exchanges gradients once per step. These are the patterns that cross the network between NVL72 racks.
Microsoft's DeepSpeed team measured communication at 17% to 47% of training iteration time on DGX H100 systems at one to four nodes, depending on model size. That is the time the GPUs sit waiting. A bigger NVLink domain lets more of the chatty traffic stay on the fast fabric. NVIDIA reports up to 1.8x more throughput per GPU when DeepSeek-R1 runs with 32-way expert parallelism across an NVL72 instead of 8-way (a vendor claim, at 100 tokens per second per user).
The 72 in NVL72 is not about having more GPUs; it is about how many GPUs can sit in one single-hop, full-bandwidth NVLink domain. An 8-GPU server has a domain of eight. An NVL72 rack has a domain of 72.
What does the 130 TB/s figure really mean?
It is an aggregate, and it is easy to misread. Each GPU still has 1.8 TB/s bidirectional. NVIDIA's 130 TB/s is that figure multiplied by 72 GPUs. The same total appears if you add up the switch side: 18 NVSwitch chips at 7.2 TB/s each is also 129.6 TB/s. What makes the rack special is not the size of the sum but that the network is non-blocking and one hop. NVIDIA describes it as all-to-all at full NVLink speed across the entire rack. No GPU pair has to share a slower uplink, so an all-to-all exchange among 72 GPUs runs much as it would among eight.
Latency matters as well as bandwidth. NVIDIA publishes no raw NVLink hop latency. The best public data point is a July 2026 study, co-authored by NVIDIA's NCCL staff, that measured a floor of about 1.4 µs for a small all-reduce on GB200 NVL72, regardless of the number of ranks. Network latency between racks varies with the fabric and tuning, and we have found no NVIDIA-published figure to quote. For the full units comparison, see 800 Gb/s InfiniBand vs 130 TB/s NVLink.
Why is the NVLink spine copper rather than fibre?
Inside the rack, NVIDIA connects the compute trays to the switch trays with a copper cable cartridge. When GB200 NVL72 launched, Jensen Huang said optics would have needed transceivers and retimers drawing about 20 kW just to drive the NVLink spine. He put the cable count at about 5,000 cables and about two miles; SemiAnalysis counted 5,184. Those are GB200 figures, but GB300 NVL72 uses the same copper-spine design. Copper only reaches a couple of metres at these speeds. That is one reason the NVLink domain stops at the rack, and why traffic between racks goes over optical InfiniBand or Ethernet. Our post on why NVIDIA uses both NVLink and InfiniBand covers that boundary.
Where does NVL72 stop, and what comes next?
- Beyond one rack. Each GPU has its own ConnectX-8 at 800 Gb/s for scale-out. NVIDIA's GB300 SuperPOD groups eight NVL72 racks (576 GPUs) per scalable unit, and its two-layer InfiniBand fabric reaches 16 units, or 9,216 GPUs.
- Power and cooling. OEMs size a GB300 NVL72 rack at about 132–142 kW, fully liquid-cooled for the compute trays. That is a data-centre project, not a server-room upgrade.
- Next generation. NVIDIA's current Vera Rubin NVL72 pages list NVLink 6 at 3 TB/s per GPU and 216 TB/s per rack. Those figures were revised down from the 3.6 TB/s and 260 TB/s announced at CES in January 2026.
For how NVL72 compares with 72 GPUs in separate servers, read NVL72 vs 72 individual GPUs. For the memory side of the rack, read 37 TB of memory and 130 TB/s NVLink.
Buying in Singapore
SourceIT does not list GB300 NVL72 racks. For data-centre-scale projects, ask us to source through our partners; we will confirm what is available and the lead time, without assuming stock. The same GB300 superchip family is available at the desk: the HP ZGX Fury G1n, MSI XpertStation WS300, Dell Pro Max with GB300, ASUS ExpertCenter Pro ET900N G3, GIGABYTE W775-V10-L01 and Supermicro ARS-511GD-NB-LCC, all price on request. Each has one GPU and no NVLink to other units. For multi-GPU servers with NVLink switches, see the Lenovo ThinkSystem SR780a V3, price on request. More options are in the AI Workstations and AI Servers collection. For InfiniBand, ConnectX or optics, which we do not list, ask us to source. For a quotation, use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.
Frequently asked questions
How many NVSwitch chips does a GB300 NVL72 have? Eighteen, in nine 1RU NVLink switch trays with two chips each. Every GPU has one NVLink 5 link to each chip, which is how all 72 GPUs connect in a single hop.
Is 130 TB/s the bandwidth between two GPUs? No. It is the aggregate of all 72 GPUs at 1.8 TB/s bidirectional each. Any one GPU sends at up to 900 GB/s in each direction.
Why does tensor parallelism stay inside the NVLink domain? Tensor parallelism combines partial results with an all-reduce at every layer. That traffic is frequent and latency-sensitive, so it runs best on NVLink. Pipeline and data parallelism tolerate the slower network between racks.
Can NVLink connect more than one NVL72 rack? NVIDIA says the NVLink 5 architecture can scale to 576 GPUs, but GB300 NVL72 is deployed as a 72-GPU domain. Racks connect to each other over ConnectX-8 at 800 Gb/s per GPU, on InfiniBand or Ethernet.
Does a DGX Station have NVSwitch? No. A DGX Station GB300 has one Blackwell Ultra GPU joined to its Grace CPU by NVLink-C2C. Two Stations connect over ConnectX-8 Ethernet, not NVLink.
SourceIT Approved
Researched and verified by the SourceIT team for Singapore buyers.
