One NVL72 Rack or Nine 8-GPU Servers? The Maths Nobody Shows You
Here is a sum that rarely appears in a sales deck. A GB300 NVL72 rack holds 72 Blackwell Ultra GPUs. An 8-GPU HGX B300 server holds eight of the same GPU family. Nine servers, therefore, also make 72.
Same GPU family. Same GPU count. So why would anyone pay for the rack? And why would anyone not? The answer is more interesting than "the rack is faster".
Seventy-two GPUs, two ways
| What | ποΈ One GB300 NVL72 rack | π₯οΈ Nine 8-GPU HGX B300 servers |
|---|---|---|
| π§ GPUs | 72 Blackwell Ultra | 72 Blackwell Ultra |
| π GPUs that share one fast NVLink domain | 72 |
8 per server |
| πΎ Memory one job can pool over NVLink | About 20 TB |
About 2.3 TB per server |
| β‘ Total power | 132 to 140 kW |
Roughly 14 to 15 kW each, so about 130 kW |
| π Power in one spot | π΄ All of it, in one rack position | π’ Spread across several racks |
| βοΈ Cooling | π§ Liquid only | π¨ Air or π§ liquid |
| βοΈ Host CPUs | Arm (NVIDIA Grace) | x86 (Intel Xeon or AMD EPYC) |
| π Can you buy part of it? | β No. It is one unit. | β Yes. Start with one server. |
Sources: HPE GB300 NVL72 product page, Servermall on DGX B300 vs DGX GB300, American Compute comparison. Per-server power is a third-party approximation and varies by vendor and configuration.
The maths nobody shows you
The electricity bill is nearly the same
Nine servers at roughly 14 to 15 kW each come to about 130 kW. The rack draws 132 to 140 kW. Choosing servers over a rack does not save meaningful power. What it changes is where the power lands. A rack concentrates it into one position that needs a liquid loop. Servers spread it across several racks that a conventional hall might cope with.
The floor space is not
An air-cooled 8-GPU server is tall. HPE's Compute XD690 is listed at 10U. Nine of them need 90U before a single switch is added, which means at least three racks. The NVL72 does the same job in one. If floor space is your scarce resource, the rack wins easily. In Singapore it often is.
The real difference is the conversation between GPUs
Inside one 8-GPU server, the GPUs share a fast NVLink connection. Between servers, traffic travels over Ethernet or InfiniBand, which is slower. In the rack, all 72 GPUs sit in one NVLink domain.
NVIDIA's own analysis says domain size matters for serving the newest large models, and that performance on mixture-of-experts models improves when moving from an 8-GPU to a 72-GPU domain. That is the case for the rack in one sentence: it is not more GPUs, it is more GPUs that behave as one.
Source: NVIDIA blog on performance per watt
Which workloads care
| Workload | Needs a 72-GPU domain? | Better fit |
|---|---|---|
| π¬ Many separate models or many small jobs | β No | π₯οΈ Servers |
| π§ Fine-tuning models that fit in eight GPUs | β No | π₯οΈ Servers |
| π Several teams sharing hardware on a schedule | β No | π₯οΈ Servers |
| π§© One very large model that does not fit in eight GPUs | β Yes | ποΈ Rack |
| π High-volume serving of large reasoning models | β Usually | ποΈ Rack |
| ποΈ Training frontier-scale models | β Yes | ποΈ Racks, plural |
The argument for servers that vendors underplay
A rack is all or nothing. You cannot buy a third of an NVL72, and you cannot run it at all until the liquid loop is commissioned. Servers arrive one at a time. You can buy two, learn what your workloads really need, and add more. If one fails, you lose an eighth of a node's GPUs, not a slice of a tightly coupled system.
Servers also run on x86 processors, which every enterprise IT team already knows. The rack's Grace CPUs are Arm. That is well supported for AI, but it is one more new thing on a project that already has plenty.
The bottom line
If your largest single job fits in eight GPUs, nine servers give you the same GPU count, the same power bill, more flexibility and an easier building. The rack earns its place only when one job needs more than eight GPUs working as one. That is a smaller group of buyers than rack-scale marketing suggests, and it is worth being honest about which group you are in.
Related on SourceIT
SourceIT Approved
Researched and verified by the SourceIT team for Singapore buyers.




