800Gbps InfiniBand vs 130TB/s NVLink: Why These Numbers Don't Tell the Whole Story

Because the two figures use different units, different scopes and different directions. 800 Gb/s for InfiniBand counts bits, for one port, in one direction. 130 TB/s for NVLink counts bytes, summed over 72 GPUs, with both directions added together. Taken at face value, the headlines suggest NVLink is about 1,300 times faster. Put both on the same basis, GB/s per GPU per direction, and they become 100 GB/s against 900 GB/s. NVLink is about 9 times faster per GPU. That is still a large gap, but it is the number that actually describes the hardware. Even then, bandwidth leaves out latency, topology and how much of the link software actually uses.

How do you put 800 Gb/s and 130 TB/s on the same basis?

Source: NVIDIA GB300 NVL72 page (accessed 6 Nov 2026) for 130 TB/s and ConnectX-8 at 800 Gb/s per GPU. NVIDIA GB200 NVL72 blog (18 Mar 2024) for "1.8 TB/s of bidirectional throughput per GPU". The conversion steps are our arithmetic. The naive headline ratio is 130 TB/s × 8 = 1,040 Tb/s, ÷ 0.8 Tb/s = 1,300.

Why is the bits-versus-bytes trap so common?

Networking speeds are quoted in bits per second (Gb/s), while GPU links and memory are quoted in bytes per second (GB/s). The two abbreviations differ only in the case of one letter, and the gap between them is a factor of 8. The confusion catches professionals as well. ABI Research wrote NVLink 4 as "900 Gbps per GPU" when NVIDIA's figure is 900 GB/s, which understates it eightfold. Any comparison between a NIC and an NVLink figure should start by dividing the NIC's bit rate by 8.

Do vendors at least count directions consistently?

No, and that includes NVIDIA. NVLink figures are bidirectional totals. PCI-SIG's headline PCIe figures are bidirectional too: 128, 256 and 512 GB/s for x16 Gen5, Gen6 and Gen7. NIC and switch port speeds are quoted per direction. Switch "capacity" is inconsistent:

• Quantum-2, with 64 × 400 Gb/s ports, is quoted as "51.2 Tb/s aggregated bidirectional", which is 25.6 Tb/s one way.
• Quantum-X800 Q3400, with 144 × 800 Gb/s ports, is quoted at 115.2 Tb/s, which is the one-way sum.
• Spectrum-4 SN5600, with 64 × 800 GbE ports, is quoted at 51.2 Tb/s, also one way, although the same manual elsewhere calls it bidirectional.

Other vendors vary as well. Google quotes its TPU Ironwood interconnect at 1,200 GB/s bidirectional per chip, or 600 GB/s each way. UALink quotes 800 Gb/s per direction for an x4 link, which is 100 GB/s each way. AMD quotes 260 TB/s for its 72-GPU Helios rack without saying which way it is counted. The safe habit is to ignore the capacity figure and recompute it yourself: ports × port speed, in one direction.

Convert every figure to GB/s, per GPU, per direction, before you compare anything.

What does 130 TB/s actually describe?

It is the sum of every GPU's injection bandwidth, meaning how fast all 72 GPUs could push data into the NVLink switches at once. No single transfer ever sees 130 TB/s, because one GPU's link tops out at 1.8 TB/s bidirectional. Nor is it the bisection bandwidth, the capacity across the narrowest cut that splits the rack into two halves, which is what limits all-to-all traffic. NVIDIA does not publish a bisection figure for NVL72. Because every GPU has one link to each of the 18 NVLink switches, a reasonable inference is 36 GPUs × 900 GB/s, or about 32 TB/s per direction across the cut. Scale-out InfiniBand fabrics, by contrast, are often deliberately oversubscribed between pods. Meta's Llama 3 cluster ran 1:7 between pods. So the per-GPU number tells you more than the rack total.

What about raw versus usable bandwidth?

Headline rates are line rates. PCIe Gen5 x16 is 64 GB/s raw per direction but about 63 GB/s after 128b/130b encoding. PCIe Gen6 is 128 GB/s raw per direction but about 121 GB/s after FLIT overhead. Protocol headers, RDMA and the communication library take more. When engineers measure collectives with NVIDIA's nccl-tests, two figures come out:

• Algorithm bandwidth is data size ÷ time.
• Bus bandwidth adjusts for the extra traffic each algorithm creates. For all-reduce it is algorithm bandwidth × 2(n−1)/n, so an 8-GPU all-reduce showing 400 GB/s of algorithm bandwidth is moving 700 GB/s of bus bandwidth.

nccl-tests says bus bandwidth "should reflect the speed of the hardware bottleneck", so it is the one to set against a link's rated speed. A vendor slide quoting one and a benchmark quoting the other can differ by almost 2x at 72 GPUs, where the factor is 2 × 71 ÷ 72 ≈ 1.97.

Is bandwidth even the right number to compare?

Often it isn't. Small messages are limited by latency, not bandwidth. NVIDIA does not publish a raw NVLink hop latency. A July 2026 study (arXiv 2607.16100, with NVIDIA NCCL co-authors) measured a floor of about 1.4 µs for small all-reduces on GB200 NVL72. It also found that each microsecond cut from all-reduce on four GB200 GPUs lowered the cost per million output tokens by about 0.9%. Network latency depends on NICs, switches, distance and software, and we found no NVIDIA-published end-to-end figure for InfiniBand, so treat any single number with care. The final gap is between peak and achieved performance. Meta's Llama 3 405B training achieved 38% to 43% model FLOPs utilisation, and ByteDance's MegaScale reported 55.2% on 12,288 GPUs. A faster link helps only as far as it raises that fraction. See 20 Petaflops GPU, Slow Model? for the other bottlenecks.

How do the Vera Rubin numbers compare?

NVIDIA's current pages give NVLink 6 at 3 TB/s bidirectional per GPU and 216 TB/s per Vera Rubin NVL72 rack. Both are revised down from the 3.6 TB/s and 260 TB/s announced at CES in January 2026, as TechRadar reported on 31 October 2026. ConnectX-9 doubles the scale-out link to 1.6 Tb/s per GPU. On the same basis, that is 1.5 TB/s against 200 GB/s per direction, so NVLink is 7.5 times faster per GPU, slightly less than Blackwell's 9x.

How should you read the next spec sheet?

1. Check bits or bytes: a lowercase "b" means bits, so divide by 8.
2. Check per port, per GPU or aggregate: divide rack and switch totals by the number of GPUs or ports.
3. Check one direction or both: halve bidirectional figures.
4. Check raw or usable: allow for encoding and protocol overhead.
5. Ask for latency and bisection, not just peak bandwidth.

For how the two fabrics divide the work, read NVLink vs InfiniBand: Why NVIDIA Uses Both. For the full ladder of links, including PCIe, read NVLink vs InfiniBand vs PCIe. For what 400G means on a desk, see 400Gbps Networking on a Desk.

Buying in Singapore

The same arithmetic applies to the networking SourceIT lists (listed 6 Nov 2026):

• The MikroTik CRS804, S$2,255.00, has four 400G ports, or 50 GB/s per port per direction. It suits a desk or lab link between a few systems, not a cluster fabric.
• The NVIDIA Spectrum-2 SN3700, S$28,523.25, has 32 × 200 GbE ports, or 25 GB/s per port per direction and 6.4 Tb/s one way across all ports.
• The HP ZGX 0.4 m QSFP112 400G DAC is S$365.00.

We do not currently list InfiniBand switches, 800G switches, ConnectX NICs or optics; ask us to source them. DGX Station GB300 systems, with dual 400G ConnectX-8, are price on request, including the MSI XpertStation WS300 and GIGABYTE W775-V10-L01. For a quotation, use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.

Frequently asked questions

Is NVLink really 1,300 times faster than InfiniBand? No. That ratio compares a 72-GPU, bidirectional, byte-based total with a single port's one-way bit rate. Per GPU, per direction, NVLink 5 is 900 GB/s and an 800 Gb/s InfiniBand port is 100 GB/s, so NVLink is about 9 times faster.

How many GB/s is 800 Gb/s? 100 GB/s per direction. Divide bits per second by 8 to get bytes per second.

Is 130 TB/s the bandwidth one GPU gets? No. It is 72 GPUs × 1.8 TB/s bidirectional, added together. One GPU gets 1.8 TB/s bidirectional, or 900 GB/s each way.

Why do switch capacity figures not match port counts? Vendors mix conventions. NVIDIA quotes Quantum-2 (64 × 400G) as 51.2 Tb/s bidirectional, but the Q3400 (144 × 800G) as 115.2 Tb/s one way. Recompute ports × speed in one direction.

Which benchmark figure should I compare with link speed? The nccl-tests bus bandwidth, which adjusts for the extra traffic of each collective. For all-reduce it equals algorithm bandwidth × 2(n−1)/n.