From 10,000 Documents to One Private AI Brain: What Can DGX Station Really Deliver?

A private, permission-aware assistant that answers questions from your own documents and cites the passages it used, for many staff at once and without the documents leaving your premises. Ten thousand documents is a small corpus for the software involved. On our rough estimate it comes to about 65 million tokens, around 127,000 searchable chunks and half a gigabyte of vectors. The DGX Station GB300 earns its place on what happens after retrieval: running a large model for dozens or hundreds of simultaneous users, re-ranking results, extracting tables and scanned pages, and running deeper research agents. What it cannot do is fix messy, outdated or badly permissioned source documents. That part of the project decides whether the "AI brain" is trusted.

How big is 10,000 documents, really?

Smaller than it sounds. The table below is an illustrative estimate. Your numbers depend on document length, file types and how you split text into chunks.

Source: SourceIT estimate, our own arithmetic. Embedding size reflects open models such as BAAI bge-m3 (1,024 dimensions, up to 8,192 tokens per input, more than 100 languages, MIT licence). We have not tested these systems.

At this size, the vector database is not the bottleneck. pgvector (a Postgres extension), Milvus Standalone (a single Docker image) and Qdrant all handle it comfortably on one machine. The engineering effort goes into extraction, chunking, permissions and answer quality.

What does the pipeline look like on a DGX Station?

NVIDIA publishes a reference design, the RAG Blueprint, that can run self-hosted on a single node with Docker Compose. Its components show what each stage involves:

• Extraction: NeMo Retriever Extraction pulls text, tables, charts and scanned text out of documents using OCR and page-element models. This step matters most for real company archives, which are full of scanned forms and tables.
• Embedding: llama-nemotron-embed-1b-v2 turns chunks into vectors. bge-m3 is an alternative for mixed English, Chinese, Malay and Tamil content.
• Search and re-ranking: Elasticsearch by default or Milvus with GPU indexing, then llama-nemotron-rerank-1b-v2 to put the most relevant passages first.
• Answering: the Blueprint's default model is nemotron-3-super-120b-a12b. Other open-weight models such as gpt-oss-120b work as well.
• Deeper research: NVIDIA's AI-Q Blueprint adds agents that clarify a question, research across sources and write a longer report, with guardrails and human-in-the-loop steps. It also supports local NIM deployment.

Every stage can run on the Station, so documents, embeddings, prompts and answers stay on hardware you control. Smaller teams can start on a GB10 system; our post Can NVIDIA GB10 Analyse Thousands of Company Documents Without Sending Them to the Cloud? covers that starting point. The Station is the step up when many people use it at once or you want a larger model.

Retrieval decides what the model reads. Permissions decide what it is allowed to read. Get both right before you worry about the model.

How many people can use it at once?

Retrieval-augmented questions carry long prompts, because the retrieved passages are sent with every question. StorageReview tested gpt-oss-120b on the ASUS ET900N G3 with exactly that kind of workload, 8,192 tokens in and 1,024 out. It measured 5,023 output tokens per second at 128 concurrent requests, about 39 tokens per second for each user. At that rate a 500-token answer takes about 13 seconds with 128 people asking at once, and far less when the system is quieter. Petronella Technology reports about 391 tokens per second for a single gpt-oss-120b user on its own DGX Station GB300. That figure is self-measured with an undisclosed setup, so treat it as indicative. For most companies with 10,000 documents, the Station will be idle much of the day. That spare capacity can go to other departments or overnight jobs, as described in The 24/7 AI Factory.

How do you stop the assistant revealing documents people shouldn't see?

Carry access rights into the index. When each chunk is stored, record which groups may see the source document, then filter search results by the user's groups before anything reaches the model. Qdrant, for example, supports filtering on stored metadata during search. Never rely on the model to "not mention" a restricted document it has already been given. Three practical rules:

1. Index from the system of record, such as your document management system or file shares, so permissions come from the same place staff already use.
2. Re-sync regularly. When someone leaves a project or a file is re-classified, the index has to follow.
3. Log queries and the documents retrieved through your gateway, so you can show later who saw what. Under PDPA section 24, logs that contain personal data need protecting too. See Can NVIDIA DGX Station Become the Foundation of Your Company's Private AI Cloud?

What will disappoint people, and how do you avoid it?

• Outdated or duplicate documents. If three versions of a policy are indexed, answers will mix them. Archive or tag superseded versions before ingestion.
• Scans and tables. Poor extraction produces confident but wrong answers. Check samples from your worst file types first.
• Questions the documents don't answer. Set the assistant to say "not found" and cite sources, rather than filling gaps from the model's general knowledge.
• Model quality. Open-weight models trail the best closed models on general benchmarks; on Artificial Analysis's index the leading open model scored 33 against 55–60 at the frontier. For retrieval-grounded questions the gap matters less, but test on your own questions.
• No evaluation set. Before launch, collect 100–200 real questions with known answers and re-run them after every change.

Back up the index and source documents as part of your normal regime. Our post Your DGX Station Could Be a Single Point of Failure covers this, and How Should Companies Charge for AI Usage? explains how to share the cost once several departments use the assistant.

Buying in Singapore

SourceIT lists all five DGX Station GB300 systems, the HP ZGX Fury G1n, MSI XpertStation WS300, ASUS ExpertCenter Pro ET900N G3, GIGABYTE W775-V10-L01 and Supermicro ARS-511GD-NB-LCC, all price on request. For the document store and backups, the Synology DiskStation DS1825+ is S$2,148.00 and the QNAP TS-873AEU-RP rackmount NAS is S$3,010.00. The D-Link DXS-1210-10TS 10GbE switch (S$1,680.00) connects them at 10GbE (all listed 5 Nov 2026). To pilot on a smaller system first, see the NVIDIA DGX Spark at S$9,300.00. For a quotation, use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.

Frequently asked questions

Do I need a DGX Station for 10,000 documents? Not for the search index, which is small. You need it when many people query at once, when you want a large model or extraction for scanned files and tables, or when the same system serves other workloads. A GB10 system can pilot the idea.

How much storage does a 10,000-document index need? On our estimate, about 0.5 GB of vectors for roughly 127,000 chunks, plus the original files. Keep originals on a backed-up NAS.

Can the assistant handle Chinese, Malay and Tamil documents? Multilingual embedding models such as BAAI bge-m3 support more than 100 languages. Answer quality in each language depends on the model you choose, so test with real documents.

How do I stop staff seeing restricted documents through the AI? Store each document's access groups with its chunks and filter search results by the user's groups before anything reaches the model.

How fast are answers? StorageReview measured about 39 output tokens per second per user with 128 concurrent long-prompt requests on gpt-oss-120b, so a 500-token answer takes around 13 seconds at that load and less when quieter.