252GB HBM3e vs 496GB LPDDR5X: The Architecture Behind NVIDIA's GB300 Superchip

The GB300 superchip has two memory tiers. The 252 GB of HBM3e sits with the Blackwell Ultra GPU and delivers 7.1 TB/s. The 496 GB of LPDDR5X sits with the Grace CPU and delivers 396 GB/s, about 18 times slower. NVLink-C2C joins them, so the GPU can reach both, but speed depends on where the data lives. For AI token generation, the rule is: whatever the model reads for every token belongs in HBM3e, and LPDDR5X is for overflow, staging and the CPU's own work. How well a deployment respects that split decides most of its real-world speed on an NVIDIA DGX Station GB300.

How do the two memory tiers compare?

Source: NVIDIA DGX Station datasheet (Aug 2026) for capacity and rated bandwidth; StorageReview's MSI XpertStation WS300 review (24 Aug 2026) for measured bandwidth; SOCAMM layout per StorageReview's HP ZGX Fury coverage (9 Sep 2026). "Best used for" is SourceIT's summary. We have not tested the system ourselves.

NVLink-C2C itself is rated at 900 GB/s, about seven times the roughly 128 GB/s of a PCIe 5.0 x16 slot counting both directions. The measured 390 GB/s shows the link is not the bottleneck: the LPDDR5X memory behind it is. Our earlier post What Does Coherent Actually Mean? explains how the GPU and CPU share one address space. Here we focus on what that means for speed.

Why does memory bandwidth govern token generation?

When a model writes an answer, it produces one token at a time. For each token, it has to read every weight it uses for that step from memory. A useful rule of thumb follows: the theoretical single-user ceiling is memory bandwidth divided by bytes read per token.

Take gpt-oss-120b. It has 117 billion parameters, but as a mixture-of-experts model it activates about 5.1 billion per token. Its expert weights are stored in a 4-bit format (MXFP4), roughly 4.25 bits per parameter with scaling, so each token reads about 2.7 GB:

• From HBM3e at 7.1 TB/s, the ceiling is about 2,600 tokens per second.
• From LPDDR5X at 396 GB/s, it would be about 146 tokens per second.

Real results sit well below the HBM3e ceiling, because attention calculations, KV cache reads, kernel launches and sampling all take time. A US managed IT provider reports about 391 tokens per second for one gpt-oss-120b user on its own DGX Station GB300 (self-measured, setup not disclosed). The direction is what matters. A model held in HBM3e runs many times faster than one read from LPDDR5X, and the 18-to-1 bandwidth ratio between the tiers is why. For dense models the gap is starker, because every parameter is read for every token; our 748GB memory explainer works through a 70B example.

Capacity tells you whether a model can load. Bandwidth tells you how fast it can speak. On the GB300, the two answers come from different memory chips.

Why do many users get more total throughput than one?

Batching changes the arithmetic. When the system serves many requests at once, it reads each weight once per step and applies it to every request in the batch. The cost of the read is shared, so total throughput rises sharply even though each user's speed changes little. StorageReview's tests on the ASUS ExpertCenter Pro ET900N G3 show this: MiniMax M2.7 produced 214 output tokens per second for one stream and 4,793 across 128 streams. gpt-oss-120b reached 9,280 output tokens per second at 128 concurrent requests.

Batching moves the limit from bandwidth to capacity. Each active request needs its own KV cache in fast memory, and that grows with context length. The 252 GB of HBM3e has to hold the model weights plus every user's cache, so a smaller model leaves more room for users. This is the main reason a model that fits comfortably in HBM3e usually serves a team better than a larger one that spills into LPDDR5X. Our post Can 748GB Really Handle a Trillion-Parameter AI Model? works through KV cache sizes.

Where does the LPDDR5X earn its place?

The slower tier is still valuable, as long as it holds the right things:

• Models that don't fit in HBM3e: inference engines can place part of a model in CPU memory. For mixture-of-experts models, less-used experts can sit in LPDDR5X while active layers stay in HBM3e. StorageReview ran GLM-5.2 this way, with about 215 GB of experts in Grace memory, at 35 tokens per second for one user. Slow, but it ran on one desk.
• Data preparation: tokenising documents, building training batches and decoding images run on the CPU, with no copy across a PCIe bus.
• Retrieval and search: vector indexes for retrieval-augmented generation, document stores and caches can live in Grace memory next to the model.
• Everything else: the operating system, containers, monitoring and the serving stack. NVIDIA reserves part of the 496 GB for the system, according to ServeTheHome.

Without the LPDDR5X tier, a 252 GB GPU would simply refuse to load anything larger. With it, the system degrades gradually instead of failing outright, which is useful for research and evaluation even when it is too slow for production serving.

How should you plan memory placement for a deployment?

1. Start with the model's size at your chosen precision. If it fits in HBM3e with room for the KV cache your users need, that is the fast path.
2. Estimate the KV cache: per-token cache size times context length times concurrent users.
3. If the model doesn't fit, decide whether a smaller model, a more aggressive quantisation or expert offload to LPDDR5X best meets your quality and speed needs. Test all three.
4. Keep the GPU's power budget intact. A high-power display card lowers the GB300's power cap; see Why Does a DGX Station Need Another Graphics Card?
5. If several teams share the GPU, partitioning splits the HBM3e between them; see Multi-Instance GPU on DGX Station.

For how these limits translate into monthly token capacity, see How Many AI Tokens Can DGX Station Actually Generate?

Buying in Singapore

All five DGX Station GB300 systems SourceIT lists share the same 252 GB HBM3e and 496 GB LPDDR5X layout: the HP ZGX Fury G1n, MSI XpertStation WS300, ASUS ExpertCenter Pro ET900N G3, GIGABYTE W775-V10-L01 and Supermicro ARS-511GD-NB-LCC, all price on request. If your models fit in less memory, a single NVIDIA RTX PRO 6000 Blackwell Workstation Edition with 96 GB is S$29,100 (listed 5 Nov 2026). For a quotation, use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.

Frequently asked questions

What is the difference between HBM3e and LPDDR5X on the GB300? HBM3e is the GPU's own memory: 252 GB at 7.1 TB/s. LPDDR5X is the Grace CPU's memory: 496 GB at 396 GB/s. Both are reachable by the GPU through NVLink-C2C, but HBM3e is about 18 times faster.

Is NVLink-C2C the bottleneck when the GPU reads CPU memory? No. NVLink-C2C is rated at 900 GB/s. StorageReview measured about 390 GB/s from Grace memory into the GPU, which matches the LPDDR5X's own 396 GB/s rating.

How does memory bandwidth affect tokens per second? For one user, the theoretical ceiling is roughly bandwidth divided by the bytes read per token. A model read from HBM3e can generate many times faster than the same model read from LPDDR5X.

Why does throughput rise with more users? Batching lets the system read each weight once and use it for every request in the batch. StorageReview measured MiniMax M2.7 at 214 tokens per second for one stream and 4,793 across 128 streams.

What should go in LPDDR5X? Overflow model weights or inactive experts, data preparation, retrieval indexes and the system software. Anything the model reads for every token should stay in HBM3e if possible.