Why Your Next AI Data Centre Needs More Than Powerful GPUs

Because a modern AI data centre is built as a system, and the GPUs are only one layer of it. In an NVIDIA GB300 NVL72 rack, 72 Blackwell Ultra GPUs depend on 18 NVSwitch chips for their NVLink connections, a ConnectX-8 network port per GPU for scale-out, a separate storage and management network, about 132–142 kW of power per rack, and liquid cooling, because air alone cannot remove that heat. Published training runs show what happens when the supporting layers fall short: even well-engineered clusters turn only about 38–55% of their peak compute into useful training work. Planning the interconnect, storage, power and cooling as carefully as the GPUs is what keeps that number at the top of the range.

What does an AI data centre need besides GPUs?

Source: NVIDIA GB300 NVL72 page (accessed 6 Nov 2026), NVL72 AI Factory reference architecture (15 Sep 2026), DGX GB300 SuperPOD reference architecture (fetched 6 Nov 2026), Supermicro, GIGABYTE, Schneider Electric and Vertiv GB300 rack specifications, and EDB's DC-CFA2 announcement (1 Dec 2025). NVIDIA publishes no single per-rack kW figure.

Why isn't GPU count the number that matters most?

Because AI training is limited by how often GPUs wait. Model FLOPS utilisation (MFU) measures how much of the theoretical compute a run actually uses. Meta reported 38–43% for Llama 3 405B on up to 16,384 H100 GPUs, Google 46.2% for PaLM, and ByteDance 55.2% for its MegaScale system on 12,288 GPUs. The gap between those figures and 100% is time spent waiting on memory, communication, stragglers and failures. Most of the supporting layers in the table above exist to shrink that gap.

Failures also scale with size. Meta's study of its A100 research cluster found a mean time to failure of 7.9 hours for a 1,024-GPU job, and listed the back-end network, the filesystem and GPUs as the main hardware causes. Over a 54-day Llama 3 snapshot, Meta logged 419 unexpected interruptions: 148 were faulty GPUs and 42 were network switches, cables or NICs. A larger cluster needs better monitoring, spares and fast recovery, not just more GPUs.

What do NVLink and NVSwitch add?

A scale-up domain where GPUs exchange data far faster than any network port allows. Each Blackwell Ultra GPU has 18 NVLink 5 links, one to each NVSwitch chip in the rack, for 900 GB/s per direction. Its ConnectX-8 network port carries 800 Gb/s, which is 100 GB/s per direction, so the NVLink connection has about nine times the bandwidth. That is why communication-heavy work such as tensor parallelism and wide expert parallelism is kept inside the NVLink domain. NVIDIA reports that spreading DeepSeek-R1's experts across 32 GPUs on NVL72 gave up to 1.8 times the per-GPU throughput of 8-way expert parallelism (a vendor claim). Our post on how GB300 NVL72 links 72 GPUs covers the switch design in detail.

The NVLink spine inside the rack is copper. For the earlier GB200 NVL72, Jensen Huang said optics would have needed about 20 kW of transceivers and retimers, and that the rack uses about 5,000 copper cables; SemiAnalysis counts 5,184. GB300 NVL72 uses the same copper-spine design.

Where do InfiniBand and Ethernet fit?

Between racks. NVIDIA's GB300 SuperPOD design gives each compute tray four ConnectX-8 adapters for an InfiniBand compute fabric that is "rail-optimized, non-blocking, full fat-tree", plus a BlueField-3 for storage and in-band traffic over Ethernet. A two-unit design (1,152 GPUs) uses 16 leaf and 8 spine InfiniBand switches; 16 units (9,216 GPUs) use 128 leaf and 72 spine. There is also a management network and an out-of-band network for the controllers. In other words, one AI data centre has at least four networks. InfiniBand vs Ethernet looks at the fabric choice, and NVLink vs InfiniBand at how the two layers divide the work.

A GPU cluster is a balanced system. Adding GPUs without matching NVLink, network, storage, power and cooling capacity mostly adds GPUs that wait.

Why does storage need planning?

Because training reads datasets continuously and writes large checkpoints regularly. A full training checkpoint, including optimizer state, is roughly 16 bytes per parameter, so a 405-billion-parameter model produces several terabytes at each save. NVIDIA's GB300 reference architecture recommends 90 GB/s read and 45 GB/s write per 576-GPU unit at its Standard tier, and 280 / 140 GB/s at its Enhanced tier. It warns that checkpointing "can introduce bottlenecks unless handled by high-bandwidth, low-latency storage systems". GPUDirect Storage, which moves data directly between storage and GPU memory without a CPU bounce buffer, is one way NVIDIA platforms raise effective bandwidth.

How much power and cooling does one rack need?

Far more than a traditional server rack. Supermicro lists its GB300 NVL72 rack at 132 kW, GIGABYTE at up to 140 kW, and Schneider Electric and Vertiv have reference designs for 142 kW. NVIDIA plans 1.2 MW for each eight-rack unit. At these densities the GPUs and CPUs are liquid-cooled through coolant distribution units (CDUs); Supermicro offers a 250 kW in-rack CDU and a 1.8 MW in-row CDU. Schneider states plainly that "air cooling alone cannot deliver" the thermal performance needed. NVIDIA also builds energy storage into the power supplies to smooth the sharp load swings of synchronised training, claiming up to 30% lower peak grid demand (a vendor claim). Meta reported power swings of "tens of megawatts" during Llama 3 training.

Liquid cooling brings its own operating tasks: coolant quality, leak detection and maintenance windows. Our post on liquid cooling maintenance covers the workstation version of those questions.

What does this mean in Singapore?

Capacity is scarce and efficiency rules are strict. Singapore paused new data centres from 2019 and resumed selectively in January 2022. The Green Data Centre Roadmap of May 2024 targeted at least 300 MW of additional capacity and a PUE of 1.3 or better. The second call for applications, launched on 1 December 2025, required PUE 1.25 and at least 50% green energy; on 21 August 2026 Digital Realty, Equinix, Keppel DC and STT GDC were each awarded 50 MW. CBRE reported Singapore colocation at US$330–475 per kW per month and vacancy of about 2% (via W.Media, July 2026). At the US$403 average, one 132–142 kW NVL72 rack would cost about US$53,000–57,000 a month in colocation before anything else.

How should a team plan its first AI infrastructure?

• Start from the workload: model sizes, training versus inference, and how many GPUs must work as one.
• Size the scale-up domain first: keep communication-heavy parallelism inside NVLink.
• Then the fabrics: compute, storage, front-end and management, with cable and optics counts. See the hidden cost of AI networking.
• Then power, cooling and space: confirm a site that can supply and cool the rack density.
• Then operations: monitoring, spares, checkpoint and restart procedures.
• Develop small first: NVIDIA positions DGX Station so that applications "can be developed and debugged locally" and "then brought to massive scale on the cloud". See from DGX Station to GB300 NVL72 and DGX Station vs traditional GPU servers.

Buying in Singapore

SourceIT supplies many of the supporting layers for departmental AI and server rooms. For power and racks: APC Smart-UPS SRT 5000VA online UPS at S$6,741.25 (the SRT range runs to S$11,052.50 for 10 kVA rackmount), and the APC NetShelter SX 42U AR3100 rack at S$2,737.50. For fast local storage: Micron 7600 PRO 15.36TB Gen5 U.2 at S$14,932 and Micron 9550 PRO 7.68TB at S$10,456. For networking: the NVIDIA SN3700 32 × 200GbE switch at S$28,523.25 (all listed 6 Nov 2026). For deskside development, DGX Station GB300 systems such as the HP ZGX Fury G1n and Dell Pro Max with GB300 are price on request, and GPU servers such as the Lenovo ThinkSystem SR680a V3 are quoted on request. We do not list NVL72 racks, InfiniBand switches or 400G/800G optics; ask us to source them. For a quotation, use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.

Frequently asked questions

What is the difference between NVLink and InfiniBand? NVLink connects GPUs inside a server or rack at up to 900 GB/s per direction per GPU on Blackwell. InfiniBand connects servers and racks at 800 Gb/s (100 GB/s per direction) per port.

How much power does a GB300 NVL72 rack use? OEM and infrastructure vendors size it at about 132–142 kW. NVIDIA plans 1.2 MW for each unit of eight racks but publishes no single per-rack figure.

Does an AI data centre need liquid cooling? At GB300 NVL72 densities, yes: the GPUs and CPUs are liquid-cooled. Lower-density GPU servers, at roughly 14 kW each, can still be air-cooled in suitable racks.

How many networks does an AI cluster have? Usually at least four: the GPU compute fabric, a storage network, a front-end or in-band network for users and orchestration, and an out-of-band management network.

Why do large training jobs fail so often? Because thousands of components must all work at once. Meta measured a mean time to failure of 7.9 hours for 1,024-GPU jobs, with GPUs, the back-end network and the filesystem as leading causes.