Rising component costs may adjust prices on selected items without prior notice. Supply constraints may add shipping time — we’ll notify you if affected. OEM costs and supply tightening — contact us to secure stock and pricing. For projects or bulk orders, choose “Request for Quotation” on any item.
Call our specialists(65) 6978 3502 Email our sales teamsales@sourceit.com.sg

Servers & Power

Why GPU Clusters Starve Without Parallel Storage: HPE Cray K3000, E2000 and C500 Explained

Organisations spend months choosing GPUs and an afternoon choosing storage. Then the cluster goes live and the GPUs spend part of every hour waiting for data to arrive. Nobody notices at first, because an idle GPU looks exactly like a busy one from across the room.

Storage is the least glamorous part of an AI cluster and one of the easiest places to waste money, in either direction. Buy too little and you starve the GPUs. Buy supercomputer storage for a two-server setup and you have overspent.

Why ordinary storage struggles

Compare 🗄️ Ordinary NAS ⚡ Parallel storage
How clients read data Through one or a few controllers From many storage servers at once
A handful of clients Fine Fine but more than needed
Dozens of GPU nodes reading together Queue forms
Scales with servers added
Skills to run it Common Specialist

A parallel file system splits data across many storage servers and lets every compute node talk to all of them directly. Add storage servers and throughput rises with them. That is why supercomputers use them. Singapore's ASPIRE 2A, for example, pairs its compute with tens of petabytes of parallel storage, including a Lustre file system.

Source: NSCC: About ASPIRE 2A

The twist: AI reads data differently

Traditional simulation writes and reads a small number of very large files, and parallel file systems were tuned for exactly that. AI workloads often do the opposite: enormous numbers of small files, read in unpredictable order. According to HPCwire's reporting, the rate at which small files can be moved in and out of GPU memory has become a bottleneck for AI inference, and classic parallel file systems are not automatically good at it.

This is why there are now two families of technology, and why HPE sells both.

Source: HPCwire: the future of storage for HPC and AI

HPE's three Cray storage systems

Compare K3000 E2000 C500
🧬 Software DAOS (object storage) Lustre (parallel file system) Lustre (parallel file system)
💽 Media All-flash NVMe All-flash, hybrid or disk All-flash or hybrid flash and disk
🎯 Best at ⚡ I/O-hungry AI: many small, fast operations 📦 Large-scale capacity and throughput 🚪 Entry point to parallel storage
🚀 Speed emphasis
📏 Scale emphasis
🏢 Typical home Large AI and mixed AI and HPC systems Supercomputers and large clusters Departmental and mid-size clusters

Sources: HPE Cray Supercomputing, Blocks & Files on the C500. The speed and scale bars are our indicative reading of HPE's positioning, not benchmark results.

K3000: built for AI's appetite

HPE describes the K3000 as the first factory-built supercomputing storage system with DAOS software embedded. DAOS is open-source object storage designed for flash, with direct integration into AI frameworks such as PyTorch and TensorFlow. HPE positions it for I/O-bound AI applications, where the aim is to keep GPUs fed. It will sit alongside the E2000 in Oak Ridge National Laboratory's next supercomputer, Discovery.

Source: Blocks & Files on ORNL Discovery

E2000: the capacity engine

The E2000 runs Lustre, the open-source parallel file system used across supercomputing. HPE highlights that it scales without software licensing by drive, capacity or system, which matters when capacity runs to petabytes.

C500: the way in

The C500 brings the same Lustre software to a smaller, less expensive design. At launch HPE pitched it as leadership-class storage technology at a fraction of the entry price, with easier operation. It is the option for a cluster that has outgrown a NAS but is nowhere near a national laboratory.

The controversial part

Vendors' GPU-efficiency numbers are their own

Storage vendors like to quote how much GPU utilisation improves once storage stops being the bottleneck. One, for example, reports customers seeing around 15% better GPU efficiency. That may well be true for those customers. It is also a figure from a company selling storage, and your result depends entirely on whether storage was your bottleneck to begin with. Measure before you buy.

Source: DDN on IO500 results

Open source cuts both ways

Lustre and DAOS are open source, which HPE presents as protection against lock-in: your data lives in a file system owned by a community, not one company. That is a fair point. The other side is that open-source parallel file systems are demanding to run, which is why most buyers pay a vendor to package and support them. You avoid licence lock-in and accept skills dependence.

Many AI teams do not need any of this

Parallel storage solves a problem that appears when many nodes read at once. One or two GPU servers do not have that problem. A good NAS with fast networking, or simply fast local NVMe drives, will serve them well for far less money.

What do you actually need?

Your setup Sensible storage
🖥️ One AI workstation or server Local NVMe plus a NAS for backup
🖥️🖥️ Two to four GPU servers A fast all-flash NAS on a fast network
🏢 A cluster of 8-GPU servers, shared by teams Entry parallel storage, C500 class
🗄️ Rack-scale AI, storage-bound training or inference Flash-first, K3000 class
🏛️ Supercomputer with simulation and AI E2000 for capacity, K3000 for speed

One test before you spend

Watch GPU utilisation during a real job. If the GPUs dip repeatedly while data loads, storage or networking is holding them back, and better storage will pay for itself. If they sit near full throughout, your storage is fine and the money belongs elsewhere.

For the smaller end of the market, see NAS vs SAN vs Object Storage: Which Does Your AI Project Need? and The AI Storage Crisis.

Related on SourceIT

SourceIT Approved

SourceIT Approved

Researched and verified by the SourceIT team for Singapore buyers.

Share This Article
Facebook

More from our blog


← Older Post Newer Post →