Rising component costs may adjust prices on selected items without prior notice. Supply constraints may add shipping time β€” we’ll notify you if affected. OEM costs and supply tightening β€” contact us to secure stock and pricing. For projects or bulk orders, choose β€œRequest for Quotation” on any item.
Call our specialists(65) 6978 3502 Email our sales teamsales@sourceit.com.sg

AI & Workstations

One NVL72 Rack or Nine 8-GPU Servers? The Maths Nobody Shows You

Here is a sum that rarely appears in a sales deck. A GB300 NVL72 rack holds 72 Blackwell Ultra GPUs. An 8-GPU HGX B300 server holds eight of the same GPU family. Nine servers, therefore, also make 72.

Same GPU family. Same GPU count. So why would anyone pay for the rack? And why would anyone not? The answer is more interesting than "the rack is faster".

Seventy-two GPUs, two ways

What πŸ—„οΈ One GB300 NVL72 rack πŸ–₯️ Nine 8-GPU HGX B300 servers
🧠 GPUs 72 Blackwell Ultra 72 Blackwell Ultra
πŸ”— GPUs that share one fast NVLink domain 72
8 per server
πŸ’Ύ Memory one job can pool over NVLink About 20 TB
About 2.3 TB per server
⚑ Total power 132 to 140 kW
Roughly 14 to 15 kW each, so about 130 kW
πŸ“ Power in one spot πŸ”΄ All of it, in one rack position 🟒 Spread across several racks
❄️ Cooling πŸ’§ Liquid only πŸ’¨ Air or πŸ’§ liquid
βš™οΈ Host CPUs Arm (NVIDIA Grace) x86 (Intel Xeon or AMD EPYC)
πŸ›’ Can you buy part of it? ❌ No. It is one unit. βœ… Yes. Start with one server.

Sources: HPE GB300 NVL72 product page, Servermall on DGX B300 vs DGX GB300, American Compute comparison. Per-server power is a third-party approximation and varies by vendor and configuration.

The maths nobody shows you

The electricity bill is nearly the same

Nine servers at roughly 14 to 15 kW each come to about 130 kW. The rack draws 132 to 140 kW. Choosing servers over a rack does not save meaningful power. What it changes is where the power lands. A rack concentrates it into one position that needs a liquid loop. Servers spread it across several racks that a conventional hall might cope with.

The floor space is not

An air-cooled 8-GPU server is tall. HPE's Compute XD690 is listed at 10U. Nine of them need 90U before a single switch is added, which means at least three racks. The NVL72 does the same job in one. If floor space is your scarce resource, the rack wins easily. In Singapore it often is.

The real difference is the conversation between GPUs

Inside one 8-GPU server, the GPUs share a fast NVLink connection. Between servers, traffic travels over Ethernet or InfiniBand, which is slower. In the rack, all 72 GPUs sit in one NVLink domain.

NVIDIA's own analysis says domain size matters for serving the newest large models, and that performance on mixture-of-experts models improves when moving from an 8-GPU to a 72-GPU domain. That is the case for the rack in one sentence: it is not more GPUs, it is more GPUs that behave as one.

Source: NVIDIA blog on performance per watt

Which workloads care

Workload Needs a 72-GPU domain? Better fit
πŸ’¬ Many separate models or many small jobs ❌ No πŸ–₯️ Servers
πŸ”§ Fine-tuning models that fit in eight GPUs ❌ No πŸ–₯️ Servers
πŸŽ“ Several teams sharing hardware on a schedule ❌ No πŸ–₯️ Servers
🧩 One very large model that does not fit in eight GPUs βœ… Yes πŸ—„οΈ Rack
πŸ“ˆ High-volume serving of large reasoning models βœ… Usually πŸ—„οΈ Rack
πŸ‹οΈ Training frontier-scale models βœ… Yes πŸ—„οΈ Racks, plural

The argument for servers that vendors underplay

A rack is all or nothing. You cannot buy a third of an NVL72, and you cannot run it at all until the liquid loop is commissioned. Servers arrive one at a time. You can buy two, learn what your workloads really need, and add more. If one fails, you lose an eighth of a node's GPUs, not a slice of a tightly coupled system.

Servers also run on x86 processors, which every enterprise IT team already knows. The rack's Grace CPUs are Arm. That is well supported for AI, but it is one more new thing on a project that already has plenty.

The bottom line

If your largest single job fits in eight GPUs, nine servers give you the same GPU count, the same power bill, more flexibility and an easier building. The rack earns its place only when one job needs more than eight GPUs working as one. That is a smaller group of buyers than rack-scale marketing suggests, and it is worth being honest about which group you are in.

Related on SourceIT

SourceIT Approved

SourceIT Approved

Researched and verified by the SourceIT team for Singapore buyers.

Share This Article
Facebook

More from our blog


← Older Post Newer Post β†’