NVIDIA DGX Station GB300 for RAG: Build a Private Enterprise Knowledge Assistant

You can build a private knowledge assistant on one DGX Station GB300, but not by running NVIDIA's RAG Blueprint unchanged. The Blueprint's published requirements are Ubuntu 22.04, three or more GPUs for its default self-hosted setup, and an embedding NIM that lists only x86 CPUs or DGX Spark's Arm CPU, doesn't list the Station's compute capability 10.3 and doesn't support MIG. NVIDIA has not published a RAG playbook for the Station, so the Blueprint as-is is unverified on this machine. The practical route is to assemble the same building blocks from documented parts: serve the language model with vLLM following NVIDIA's Station playbook, add a small embedding and reranking model, store vectors in pgvector or another vector database you have tested on Arm, and feed it from your file shares. The model side fits easily in the 252 GB of HBM3e; the real work is ingestion, permissions and evaluation. SourceIT has not built this stack on a Station; the steps below follow NVIDIA, vLLM, pgvector and OWASP documentation.

What does each RAG component need, and what is documented for the Station?

Source: NVIDIA RAG Blueprint support matrix (GitHub, main branch, accessed 9 Nov 2026); NVIDIA embedding NIM support matrix (updated 17 Aug 2026); NVIDIA CUDA GPUs page; DGX Station vLLM playbook (build.nvidia.com/station); pgvector GitHub. "Not confirmed" means no primary source was found as of 9 Nov 2026. SourceIT has not tested these components on a Station.

Why not just run the NVIDIA RAG Blueprint?

Because its documented targets are different machines. The Blueprint's default Docker deployment says the LLM "requires 2 GPUs (FP8 TP2)" and lists three H100, B200 or RTX PRO 6000 GPUs; the Station has one GB300. Its operating-system line says Ubuntu 22.04, while the Station ships Ubuntu 24.04. The embedding NIM's optimised profiles cover compute capability 12.0, 10.0, 9.0 and 8.9, not 10.3. A comparable attempt on DGX Spark, reported by Classmethod in August 2026, had to rebuild x86-only images and could not get the OCR NIM working. The Blueprint is still the best reference design for what a production RAG pipeline contains. Use it as a parts list and adapt, and be ready for some components not to run as shipped.

How do you split the GB300 between generating, embedding and reranking?

The sizes leave plenty of room. Nemotron 3 Super 120B NVFP4 takes about 80 GB of repository storage. NVIDIA's support matrix puts the 1B embedding model at about 3–7 GiB depending on precision. Together that is well under the 252 GB of HBM3e, leaving more than 100 GB for the LLM's KV cache (our inference; NVIDIA has published no Station RAG sizing). The simplest layout runs everything as separate containers on the full GPU, with vLLM's memory share capped so the embedding and reranking services have room. Follow our vLLM deployment guide for the serving side, including pinning containers to the GB300.

MIG looks attractive for separating services, but the embedding NIM says MIG is not supported, and MIG layouts do not survive a reboot. If you do slice the GPU, test every component inside its slice. Remember that MIG divides only the HBM; the 496 GB of LPDDR5X and the CPU cores are shared.

How do you get company documents into it?

Start with one source you control, usually a department file share on the NAS, and build out from there. Mount it read-only on the Station, as described in our NAS integration guide, and keep the source of truth on the NAS rather than copying archives to local NVMe. The Blueprint's ingestion handles multimodal PDFs with tables, charts and images. If its extraction NIMs won't run, parse documents on the Grace CPU with an open-source parser, and record which documents failed to extract.

No published figure tells you the right chunk size for your documents. Pick a starting point, then adjust using an evaluation set. Schedule re-indexing for when files change, for example a nightly job for new or modified files, and log what was indexed. A large first index can run outside office hours, so it doesn't compete with users for the GPU.

Most RAG projects succeed or fail on retrieval and permissions, not GPU speed. One GB300 has more model capacity than a knowledge assistant needs; spend the effort on clean sources and access control.

How do you stop the assistant showing documents people shouldn't see?

OWASP's 2025 guidance on vector and embedding weaknesses (LLM08) recommends "permission-aware vector and embedding stores", "strict logical and access partitioning of datasets", accepting data "only from trusted and verified sources" and "detailed immutable logs of retrieval activities". In practice, store each document's access group with its chunks, filter retrieval by the signed-in user's groups before anything reaches the model, and keep separate collections or databases for sensitive departments. Our earlier post on permission-aware search explains why this is the real project, and our security hardening checklist covers the host itself.

On-premises RAG keeps the documents on your machine, which removes the Personal Data Protection Act's transfer-limitation question for overseas processing, but your protection and breach-notification obligations still apply. This is general information, not legal advice; confirm requirements for your situation with your legal adviser.

How do you know it is ready for users?

Write an evaluation set before you build: real questions from the department, the document that holds each answer, and the answer a reviewer accepts. Run it after every change to chunking, the embedding model or the prompt. Track three things: whether the right document was retrieved, whether the answer is correct and cited, and whether any answer drew on a document the asker shouldn't see. Release to a small pilot group first, and keep a feedback button on every answer. Our 10,000-document private AI brain post covers the sizing and the pitfalls that disappoint users, and where company data goes when using AI sets out the data questions management will ask.

Buying in Singapore

SourceIT lists DGX Station GB300 systems from six OEMs, supplied through each OEM's channel: the ASUS ExpertCenter Pro ET900N G3, Dell Pro Max with GB300, HP ZGX Fury G1n, MSI XpertStation WS300, GIGABYTE W775-V10-L01 and Supermicro ARS-511GD-NB-LCC, all price on request. For the document store next to it, the Synology RackStation RS822RP+ is S$2,086.80 (listed 9 Nov 2026); prices change, so ask for a current quotation. Plan capacity with our storage planning worksheet. Use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.

Frequently asked questions

Does NVIDIA's RAG Blueprint run on a DGX Station GB300? It isn't documented for the Station. The Blueprint lists Ubuntu 22.04, multi-GPU defaults and an embedding NIM that names only x86 or DGX Spark CPUs, so expect to adapt it.

Which language model should a Station RAG assistant use? A model NVIDIA documents for the Station's vLLM playbook, such as Nemotron 3 Super 120B NVFP4, is a sensible start. It is also the Blueprint's default LLM family.

Which vector database works on the Station? pgvector runs inside PostgreSQL and supports HNSW indexes; use halfvec for 2,048-dimension embeddings. Check arm64 support before choosing Milvus or another engine.

Can embedding, reranking and the LLM share one GB300? Yes, as separate services on the full GPU. The models need far less than 252 GB, but the embedding NIM doesn't support MIG, so test any sliced layout.

Does on-premises RAG make us PDPA-compliant? No. Keeping data local removes overseas-transfer questions, but protection and breach-notification obligations still apply. Get legal advice for your situation.