128GB RAM Sounds Crazy. Until You Start Running Local AI.

A 128 GB computer sounds like an overspecified indulgence until you load your first 70B-parameter language model and watch it refuse to fit on a 32 GB graphics card. Local AI is a memory problem before it is a compute problem. This post shows the arithmetic step by step, explains why unified memory on NVIDIA's GB10 (the ASUS Ascent GX10 and DGX Spark) and on RTX Spark laptops changes what a desk or a bag can run, and where a discrete GPU still wins. For machine-by-model lists, see our best PC to run LLMs locally and best laptop to run LLMs locally guides; this post explains why the numbers come out as they do.

The arithmetic for one 70B model

Memory for weights is parameters x bytes per weight: 2 bytes at FP16, 1 byte at FP8, 0.5 bytes at FP4, plus roughly 10 per cent at FP4 for quantisation scales. To that you add the KV cache, which we calculate in the next section; the table uses a 32,768-token context held at FP16. Example model: Llama 3.3 70B.

Reserve memory for the operating system and the model runtime as well; on a unified-memory machine they share the pool, and our laptop guide plans 8 to 12 GB for Windows. Even at FP4, a 70B model outgrows a 32 GB card before it reads a word of your document.

The KV cache: memory that grows with every word

Weights are fixed; the KV cache is not. For every token in the context, each layer stores a key and a value vector so the model does not recompute them. The size per token is 2 (key and value) x layers x KV heads x head dimension x bytes per value. Llama 3.3 70B's published configuration lists 80 layers, 8 KV heads and a head dimension of 128, so at FP16:

2 x 80 x 8 x 128 x 2 bytes = 327,680 bytes, about 0.33 MB per token.

Multiply by context length. A 32,768-token session (a long contract or a day of agent history) is 32,768 x 327,680 bytes = about 10.7 GB. The model's full 131,072-token context is about 42.9 GB, more than the FP4 weights themselves. Five people each holding a 32k-token conversation is about 54 GB of cache. Storing the cache at FP8 halves each figure. This is why a model that "fits" on a spec sheet runs out of memory once a team uses it.

Put the two together on 128 GB: FP4 weights (about 39 GB) plus three 32k-token sessions (about 32 GB) plus 12 GB for the OS and runtime is about 83 GB. On a 64 GB machine the same 70B model fits with one 32k-token session (39 + 10.7 + about 10 GB for the OS is roughly 60 GB); on a 32 GB GPU it does not start.

Unified memory vs VRAM: why the shape of the pool matters

A desktop graphics card such as the GeForce RTX 5090 has 32 GB of GDDR7 on a 512-bit bus, physically on the card. System RAM is a separate pool on the other side of the PCIe slot. When a model is bigger than VRAM, runtimes can offload some layers to system RAM, but those layers then run at CPU-memory speed and the whole model slows to match. Capacity on a discrete GPU is a hard wall.

NVIDIA's GB10 takes the other approach. NVIDIA's DGX Spark specification describes 128 GB of LPDDR5X as "coherent unified system memory" on a 256-bit interface at 273 GB/s: the Grace CPU and the Blackwell GPU address the same pool, so the GPU can use nearly all of it for weights and cache. RTX Spark laptops apply the same idea in a laptop, with up to 128 GB of unified LPDDR5X that CUDA addresses directly.

The trade-off is speed per gigabyte. GDDR7 on a discrete card is much faster than LPDDR5X, so a model that fits comfortably in 32 GB will usually generate tokens faster on the graphics card. Unified memory wins on what loads at all.

Bandwidth: what 128 GB does not give you

Once a model fits, memory bandwidth sets its speed. Each generated token reads the active weights once, so the single-stream ceiling is roughly bandwidth divided by model size. A 39 GB FP4 70B model on a 273 GB/s GB10 has a ceiling of 273 / 39 = about seven tokens per second for one user, and real results sit below that. Batching several users raises total throughput.

Mixture-of-experts models are the reason unified memory and modest bandwidth work well together. They hold many parameters but read only the active experts for each token: the default model NVIDIA's NemoClaw playbook served on a 128 GB DGX Spark is Qwen3.6-35B-A3B at NVFP4, where "A3B" denotes about 3B active parameters. Memory for the whole model, bandwidth for the active slice: a good fit for a big, slower pool.

What 128 GB lets you run, by the same arithmetic

70B at FP8 with long context: 70 GB weights plus 10.7 GB per 32k session; higher quality than FP4 and still inside 128 GB. 120B-class at FP4: 120 x 0.5 x 1.1 = about 66 GB, leaving roughly 60 GB for cache and the OS. 200B-class at FP4: 200 x 0.5 x 1.1 = about 110 GB, the practical ceiling, which matches NVIDIA's "up to 200 billion parameters" for one 128 GB unit. Beyond that, two GX10s over ConnectX-7 pool 256 GB, which NVIDIA rates for models up to 400B parameters; our GX10 clustering guide explains the set-up. A single box that holds more is the GB300 class: the ASUS ExpertCenter Pro ET900N G3 has 748 GB of coherent memory and is priced on request; see what you can run on a DGX Station GB300.

Your 128 GB options in Singapore

Desk unit: the ASUS Ascent GX10 1 TB at S$7,200, with 2 TB at S$8,050 and 4 TB at S$9,850; the MSI EdgeXpert 4 TB at S$6,895; and the NVIDIA DGX Spark 4 TB at S$8,600. All carry the same GB10 and 128 GB; our GB10 buyer's guide covers the differences. Laptop: the MSI Prestige N16 Flip AI+ RTX Spark at 128 GB for S$10,854.12 or 64 GB for S$6,331.28; memory is soldered, so buy the size you will need. Discrete GPU: for models of about 13B and under, a CUDA desktop such as the ASUS ROG NUC with a 16 GB RTX 5080 is faster per token.

Whether a business needs one at all is covered in do you need an AI supercomputer? and who should buy a S$10,000 AI computer. For cost against an API, see when running AI locally beats cloud; for the agents that consume long contexts, your next AI employee might be a computer on your desk.

Buying a 128 GB local AI machine in Singapore

SourceIT is an authorised ASUS reseller with official ASUS Singapore warranty on the Ascent GX10, and stocks the GB10 and RTX Spark machines above. Tell us the model, precision, context length and number of users, and we will run the same arithmetic with you (or use the AI hardware calculator), then deliver a GX10 commissioned: updated, networked and serving. Education and NVIDIA Inception pricing may apply on the GX10. Use Request for Quotation on the product page, call (65) 6978 3502, or email Sales@sourceit.com.sg; we are GeBIZ and Vendors@Gov registered. For the wider line-up, see the ASUS Agent Computer guide.

Frequently asked questions

How much memory does a 70B model need? Weights are 140 GB at FP16, 70 GB at FP8 and about 39 GB at FP4, plus about 10.7 GB of KV cache per 32,768-token session at FP16 and memory for the OS. A 128 GB unified-memory machine runs it at FP4 or FP8; a 32 GB GPU cannot load it.

What is the KV cache? The stored keys and values for every token in the context. For Llama 3.3 70B it is about 0.33 MB per token at FP16, so long documents and many users add tens of gigabytes.

What is the difference between unified memory and VRAM? VRAM is a separate pool on the graphics card; unified memory is one pool the CPU and GPU share. Unified memory lets the GPU use far more capacity; VRAM is faster per gigabyte.

Is a 32 GB RTX 5090 better than a 128 GB GB10 for local AI? For models that fit in 32 GB it is usually faster. For 70B-class models and long contexts it cannot load what the GB10 runs.

Is 64 GB enough? For a 70B model at FP4 with one long session, yes. For FP8, 120B-class models or several users, choose 128 GB.

Can I add memory later? No. GB10 systems ship with a fixed 128 GB and the MSI RTX Spark laptops use soldered LPDDR5X; to go beyond 128 GB you cluster GB10 units or move to the GB300 class.