RTX 5090 Has 32GB VRAM. Why Are AI Developers Still Complaining?

Because 32 GB fills up faster than it sounds. The RTX 5090 is one of the fastest single cards for local AI, with 1,792 GB/s of memory bandwidth, but that memory has to hold the model's weights, a cache that grows with every token of context, and working buffers. A 32B model at 4-bit takes about 20 GB before you type anything. A 70B model at 4-bit is about 42.5 GB and doesn't fit at all. Add long documents, several users or fine-tuning and the ceiling arrives quickly. Because GeForce cards have no NVLink, a second 5090 gives you two 32 GB pools, not one 64 GB one.

What actually fits in 32 GB?

Source: Hugging Face model file listings (unsloth GGUF repositories and OpenAI's gpt-oss checkpoints), accessed 6 Nov 2026; llama.cpp gpt-oss guide; hardware-corner (Oct 2025). "Fits" assumes about 1.5–3 GB of headroom for the CUDA context and a modest cache.

The weights are only the starting point. Every extra token of context, every additional user and every training step needs more memory on top.

Why does a long conversation eat memory?

Language models keep a key-value (KV) cache of every token they've read, so they don't recompute it. Its size is roughly 2 × layers × KV heads × head size × bytes per value × tokens. For Llama 3.1 8B that works out to 131,072 bytes per token, so a 128K-token context needs about 17.2 GB of cache at 16-bit precision, more than three times the 4.9 GB of 4-bit weights. For Qwen3-32B it is about 262,144 bytes per token: 8.6 GB at 32K.

The measurements match. Hardware-corner found Qwen3 32B at 4-bit maxed out at about 45,000 tokens of context in a 5090's 32 GB, while Qwen3 8B reached 131,000. Speed also drops as context grows: Qwen3 32B generated 61.4 tokens per second at 4K context and 43.8 at 32K on the same card.

There are ways to stretch it:

• Quantise the cache. Ollama's documentation says q8_0 uses "approximately 1/2 the memory of f16 with a very small loss in precision", and vLLM's FP8 KV cache allows "approximately double" the cache space.
• Check the default. Ollama uses a 4,096-token context window unless you raise it, so a model can appear to "forget" a long document simply because the setting is small.
• Pick a model with a lighter cache. gpt-oss-20b ran at its full 131K context in about 15 GB.

What happens when several people share one card?

A single 5090 serves small models to many users well. In a January 2026 study of consumer Blackwell cards (Knoop and Holtmann, vLLM 0.12), one 5090 delivered 4,844 tokens per second across 64 concurrent users on gpt-oss-20b. The trouble starts with bigger models and long prompts. With Gemma 3 27B at 4-bit, four users sending 32,000-token documents waited about 50 seconds for the first word, because the cache for four long contexts competes with the weights for the same 32 GB. That's the practical form of the complaint: the card is fast, but it runs out of room to be fast for more than a few people at once.

Can you fine-tune on 32 GB?

Within limits. Unsloth's published minimums put an 8B QLoRA at 6 GB and an 8B 16-bit LoRA at 22 GB, a 32B QLoRA at 26 GB and a 70B QLoRA at 41 GB. So a 5090 can fine-tune up to about 32B with QLoRA, or 8B in 16-bit, but not a 70B model or any 14B-plus model in 16-bit LoRA. Our post on LoRA and QLoRA fine-tuning on GB10 covers the larger-memory route.

The RTX 5090 is fast enough for almost anything that fits in 32 GB. Developers complain because the interesting models, long contexts and multi-user workloads increasingly don't.

Why not just add a second RTX 5090?

You can, but it doesn't merge the memory. GeForce RTX 50 cards have no NVLink, and an NVIDIA forum moderator has said peer-to-peer transfers aren't supported on them, so the cards talk through system memory over PCIe. The model has to be split. DatabaseMart measured two 5090s running Llama 3.3 70B at 4-bit at 26.85 tokens per second in Ollama, and a 110B model at only 7.22. Two cards also mean about 1,150 W of GPU power, and NVIDIA's GeForce driver licence says it is "not licensed for datacenter deployment", which matters if the plan is a shared server. We explain the PCIe side in no NVLink bridge on RTX PRO 6000, which applies equally to GeForce cards.

What are the alternatives when 32 GB isn't enough?

• A GB10 system such as DGX Spark or ASUS Ascent GX10 has 128 GB of unified memory, but only 273 GB/s of bandwidth. It runs gpt-oss-120b entirely in memory at about 59 tokens per second (llama.cpp, Feb 2026), where a 5090 manages roughly 10–30 with experts offloaded, depending heavily on system RAM. On models that fit in 32 GB, the 5090 is about 4–6.5 times faster. See RTX 5090 vs DGX Spark and our earlier GB10 vs RTX 5090.
• An RTX PRO 6000 Blackwell uses the same GB202 chip and bandwidth with 96 GB on one GPU, plus ECC and a licence that allows data-centre use. See RTX 5090 vs RTX PRO 6000 and who actually needs 96 GB.
• Smaller or sparser models. Mixture-of-experts models such as Qwen3 30B-A3B run fast on a 5090 because only a few billion parameters are active per token.

Buying in Singapore

RTX 5090 cards are not in stock at SourceIT at the time of writing; ask us about availability and a quotation. For more memory, the ASUS Ascent GX10 (128 GB, 1 TB) is S$10,350, the NVIDIA DGX Spark 4 TB is S$9,300, the RTX PRO 5000 Blackwell (48 GB) is S$14,971.20 and the RTX PRO 6000 Blackwell Workstation Edition (96 GB) is S$29,100 (all listed 6 Nov 2026). Prices change; ask for a current quotation. Use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.

Frequently asked questions

Can an RTX 5090 run a 70B model? Not comfortably. Llama 3.3 70B at 4-bit (Q4_K_M) is about 42.5 GB, more than the card's 32 GB. It needs a second GPU, offloading to system memory or a heavily quantised 2-bit version.

What is the biggest model that fits on an RTX 5090? In practice, about 32B parameters at 4–6 bit, such as Qwen3-32B (19.8 GB at Q4_K_M), with room for roughly 45,000 tokens of context. Mixture-of-experts models such as gpt-oss-20b fit with their full context.

Why does my model slow down with long documents? The KV cache grows with every token. On a 5090, Qwen3 32B fell from 61.4 to 43.8 tokens per second between 4K and 32K context, and once the cache exceeds VRAM it spills or fails.

Do two RTX 5090s give 64 GB of VRAM? No. They give two separate 32 GB memories. The model must be split across them over PCIe, as GeForce cards have no NVLink.

Can I fine-tune on an RTX 5090? Yes, up to about 32B with QLoRA (Unsloth lists 26 GB) or 8B with 16-bit LoRA (22 GB). A 70B QLoRA needs about 41 GB.