Sometimes, but probably not yours. The claim is true for AI training spread across many servers. Published studies show GPU-to-GPU communication taking 17–47% of training time in those setups, and badly tuned networks wasting much of a large cluster's capacity. A typical company AI project in Singapore looks different: one to eight GPUs, or a couple of compact GB10 computers, serving a chatbot or document search to staff. There, the first limits are GPU memory and memory bandwidth. The network matters in three narrower places: how fast you can load models and data, how users reach the service, and, only if you split one model across machines, the link between those machines. Get those right and a faster switch adds little.
How fast are the links an SME AI project actually uses?
| Link | Speed per direction | Where it shows up in an SME project |
|---|---|---|
| 1 GbE | 0.125 GB/s | Office LAN; fine for users' chat traffic, slow for moving models |
| 10 GbE | 1.25 GB/s | NAS to workstation; GB10 boxes have a 10 GbE port |
| 100 GbE (RoCE) | 12.5 GB/s | Small 2–4 node cluster, fast shared storage |
| 200 Gb/s (GB10 ConnectX-7 port) | 25 GB/s (measured ~27 GB/s across ports) | Linking two to eight GB10 computers |
| PCIe 5.0 x16 | ~63 GB/s (measured GPU-to-GPU ~46 GB/s) | GPUs inside one workstation |
| NVLink 5 | 900 GB/s | Inside data-centre GPU systems only |
Source: CALC (illustrative) converting Gb/s to GB/s by dividing by 8, with no protocol overhead. Measured figures: GB10 NCCL all-reduce reports in community and NVIDIA testing (2026); two-card RTX PRO 6000 peer-to-peer result from Level1Techs (17 Jul 2025). NVLink figure from NVIDIA specifications.
Where does the "networking beats GPUs" claim come from?
From very large training runs. Microsoft's DeepSpeed team measured tensor-parallel communication at 17–43% of iteration time when training a 13-billion-parameter model across one to four DGX H100 servers (arXiv, September 2024). DeepSeek's V3 report described a roughly 1:1 ratio of compute to communication for its cross-server expert parallelism. Meta said its two 24,576-GPU clusters performed "initially poor and inconsistent" until it tuned scheduling, routing and NCCL, after which they ran above 90% efficiency (Meta Engineering, March 2024).
Those findings are real, and we cover what they mean at data-centre scale in Why Your Next AI Data Centre Needs More Than Powerful GPUs and Could Your S$1 Million AI Cluster Be Wasting Money Because of Poor Networking?. They do not transfer to a single box. Not every study points at the network either: a ByteDance study (OSDI 2025) found most slow training jobs were caused by uneven computation, not network delays.
What limits a small AI project instead?
For inference, which is most company AI, generating each token is limited by memory bandwidth. NVIDIA's own technical guidance describes the decode phase as memory-bound. That is why the first question is whether the model fits in GPU memory and how fast that memory is, not how fast the office switch is.
The network enters at three points:
- Loading models and data. As an illustration (CALC), a model of about 65 GB, such as gpt-oss-120b, loads from a local Gen4 NVMe drive at about 7 GB/s in roughly 9 seconds. Over 10 GbE from a NAS at about 1.1 GB/s it takes about a minute; over 1 GbE, about 10 minutes. If you reload models often or feed large document sets into retrieval, 10 GbE between storage and the AI computer is worth having.
- Serving users. Chat responses are small. Ordinary office Ethernet carries them easily; latency and Wi-Fi quality matter more than raw bandwidth.
- Splitting one model across machines. This is the only point where SME projects hit the data-centre problem. Tensor parallelism needs an exchange at every layer, which is why it is normally kept inside one box. Linking GB10 computers over their 200 Gb/s ports works, but scaling is uneven: NVIDIA reported an 8-node GB10 setup reaching 4.43 times the decode speed of one node (22 Oct 2026).
Buy memory first. Upgrade the network when you move models around often, or when one model no longer fits in one box.
When does a small project need serious networking?
Fine-tuning across two GPUs or two computers. As a rough CALC that ignores overlapping compute with communication, syncing the full gradients of an 8-billion-parameter model (about 16 GB in BF16) between two GPUs takes about 1.3 seconds per step over 100 GbE, 0.64 seconds over a 200 Gb/s link and 0.35 seconds over PCIe peer-to-peer. LoRA fine-tuning syncs only small adapter weights, so the gap mostly disappears.
Many users, many replicas. Running separate copies of a model on separate GPUs needs almost no GPU-to-GPU traffic. Community testing has found two separate copies serving close to twice the throughput of one. For many concurrent users this is often better than splitting one model across machines.
Models too big for one box. Here the link between machines decides speed. Expert-parallel serving over PCIe has been reported as dramatically slower than keeping the model on one NVLink system (vLLM forum, April 2026). At that point, consider whether a larger single system or a cloud service suits better.
A cheap fast switch is not free to run. RoCE needs flow control configured, and community reports of link drops on small GB10 clusters show the extra maintenance involved. Our guide to clustering two ASUS Ascent GX10 units covers the practical setup.
What should you check before buying switches?
- Does the model fit in one GPU or one box? If yes, prioritise GPU memory, not the network.
- Where do models and documents live? If on a NAS, put 10 GbE between the NAS and the AI computer. Our 2.5GbE sweet spot post covers the step-up ladder for the rest of the office.
- How many people will use it at once? More users usually means more replicas, not a faster interconnect.
- Will you fine-tune? Full fine-tuning across machines needs fast links; LoRA rarely does.
- Who will run it? Faster networks add configuration work. Our post on buying AI servers before an AI strategy covers planning first.
Storage is the other half of the data path. Why fast SSDs are getting more expensive, and when they really are the bottleneck, is covered in the AI storage crisis.
Buying in Singapore
For storage-to-AI links, the Synology E10G18-T1 10GbE card is S$202.80 and the D-Link DXS-1210-10TS 8-port 10GBASE-T switch is S$1,680. For small clusters, the Synology dual-port 25GbE card is S$547.20 and the MikroTik CRS804 4×400G switch is S$2,255. For the AI computer itself, the ASUS Ascent GX10 128 GB / 1 TB is S$10,350 (all listed 8 Nov 2026). See also our RUCKUS ICX switches. Prices change; ask for a current quotation. To size a project, use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.
Frequently asked questions
Does my AI server need a 100GbE switch? Usually not if the model runs in one box. You need fast links when you split one model across machines, run full fine-tuning across machines or move large models and data often.
Is 10GbE worth it for local AI? Yes if models or documents come from a NAS. As an illustration, a 65 GB model takes about a minute to load over 10 GbE and about 10 minutes over 1 GbE.
Does a faster network make chat responses faster? Rarely. Token generation is limited by GPU memory bandwidth. User traffic for a chatbot is small.
Can I link two GB10 computers to run bigger models? Yes, over their 200 Gb/s ConnectX-7 ports. Scaling is uneven: NVIDIA reported 8 nodes reaching about 4.4 times one node's decode speed.
Is InfiniBand needed for a small AI cluster? No. Small clusters normally use Ethernet with RDMA. At data-centre scale, both InfiniBand and Ethernet are in wide use.
