Deploying vLLM on NVIDIA DGX Station GB300: A Practical Enterprise Guide

Start from NVIDIA's own "Serve LLMs with vLLM on One Device" playbook for DGX Station, not a generic server recipe. It runs the upstream vllm/vllm-openai container, pins vLLM to the GB300 by UUID so it doesn't land on an RTX PRO display card, and serves an OpenAI-compatible API on port 8000. Four decisions then shape an enterprise deployment: which container image you standardise on, which quantised checkpoint you load (NVFP4 is NVIDIA's recommended format on this chip), how much of the 252 GB of HBM3e you give vLLM, and how you put the endpoint behind authentication. vLLM sizes itself against HBM, not the full 748 GB of coherent memory; the 496 GB of LPDDR5X is only used if you switch on offload, and it is much slower. SourceIT has not run this deployment itself. Everything below follows NVIDIA's DGX Station documentation and playbooks, the vLLM docs and named third-party reviews.

What decisions does a vLLM deployment on a DGX Station involve?

Source: NVIDIA DGX Station vLLM playbook (build.nvidia.com/station/vllm, body last updated 19 Oct 2026), NVIDIA NVFP4 quantization playbook (updated 21 Oct 2026), vLLM engine-arguments and OpenAI-compatible server docs (accessed 9 Nov 2026). CALC = our arithmetic. Facts as of 9 Nov 2026; SourceIT has not tested this configuration.

Which vLLM container should you use on the GB300?

There are two documented routes, and both are fine as long as you choose one and pin it. NVIDIA's Station vLLM playbook uses the upstream Docker Hub image vllm/vllm-openai, at tag v0.31.0 for Nemotron 3 Super and a model-specific qwen38-flash-next tag for Qwen3.8 Flash-Next. NVIDIA's NVFP4 quantization playbook, which lists DGX Station as supported, uses the NGC image nvcr.io/nvidia/vllm:25.12.post1-py3 and tells you to "use the image for your hardware platform". The shipping software stack already includes vLLM.

Two details matter on this chip. NVIDIA's CUDA GPUs page lists the DGX Station GB300 as compute capability 10.3 (sm_103), not 10.0 like B200. And the NGC vLLM 26.10 release notes say SM100/SM103 is supported but don't name GB300, DGX Station or arm64. Treat "Blackwell supported" in any release note as a prompt to test, not a guarantee.

How do you make sure vLLM runs on the GB300?

Many Stations are fitted with an RTX PRO card for the desktop, because the GB300 has no display output. That card is also a CUDA device. NVIDIA's playbook troubleshooting page describes an out-of-memory error "on a ~95 GiB GPU while loading DeepSeek" caused by --gpus all letting vLLM pick the RTX PRO card. The playbook's fix selects the GB300 by UUID:

GB300_UUID=$(nvidia-smi --query-gpu=uuid,name --format=csv,noheader | \
awk -F, 'tolower($2) ~ /gb300/ {gsub(/[[:space:]]/, "", $1); print $1; exit}')
: "${GB300_UUID:?No GB300 found}"

You then replace --gpus all with --gpus "device=${GB300_UUID}" in the container command. NVIDIA's development guide adds that a boot service places the GB300 at CUDA index 0 and the RTX PRO card at 1, but "containers do not inherit them", so set the visible devices explicitly. The playbook generates the full docker run line from its recipe page; copy it from there rather than from a blog.

How much memory does vLLM really get?

vLLM allocates weights and KV cache in GPU memory. NVIDIA's DGX Station development guide says cudaMalloc "still allocates memory exclusively in GPU VRAM (HBM on Blackwell)", and that in the default memory mode, system memory is not migrated into GPU memory. In practice vLLM works within the 252 GB of HBM3e, and its default --gpu-memory-utilization of 0.92 gives it about 232 GB (CALC) before activations and CUDA graphs.

The LPDDR5X only comes into play when you turn on offload, for example --cpu-offload-gb for weights or a KV offload setting. StorageReview's MSI XpertStation WS300 review (24 Aug 2026) shows the trade-off. MiniMax M3 NVFP4, with only 21 GB of experts offloaded, generated 197 tokens per second for a single user. GLM-5.2 NVFP4, with 216 GB offloaded, managed 36 tokens per second single-stream, and StorageReview noted that generation was limited by "moving the offloaded experts". Our 748 GB memory explainer covers why the two pools behave so differently.

Rule of thumb: fit the model and the hot KV cache in HBM. Treat the LPDDR5X as overflow for models that won't fit, and expect single-user speed to fall when you use it.

Which model and precision should you start with?

Begin with one of the checkpoints NVIDIA names for the Station: Nemotron 3 Super 120B A12B NVFP4 (about 80 GB of repository storage on Hugging Face) or Qwen3.8 Flash-Next NVFP4. NVIDIA's NVFP4 playbook claims the format cuts memory "roughly 3.5x vs FP16 and 1.8x vs FP8" with accuracy usually within 1% of FP8; that is a vendor claim, and NVIDIA itself says to evaluate per use case.

Then size the KV cache against your real traffic. As an illustration (CALC, from model configs, before activations): Llama 3.3 70B in FP8 uses about 71 GB, leaving roughly 161 GB of the 0.92 budget. With a BF16 KV cache that holds about 0.49 million tokens, or about four full 131,072-token sessions; with --kv-cache-dtype fp8 it is roughly double. NVIDIA's playbook warns that setting a long --max-model-len with --max-num-seqs 8 "does not guarantee eight full-length requests fit". Set limits from your logs. For measured throughput across models, see our DGX Station tokens-per-second post.

How do you check the server and connect users safely?

NVIDIA's playbook health checks are two calls:

curl --fail-with-body -i http://localhost:8000/health
curl --fail-with-body -sS http://localhost:8000/v1/models

For a single developer, the playbook uses an SSH tunnel (ssh -N -L 8000:127.0.0.1:8000 user@dgx-host) so the API never leaves the machine. For a department, require an API key with vLLM's --api-key option, keep the bind address local or firewalled, and publish the endpoint through a gateway that handles sign-in, budgets and audit logs, as described in our private AI cloud post. Add GPU telemetry from day one with DCGM, which is pre-installed; our DCGM, Prometheus and Grafana guide shows the setup. Run it as a managed service with a pinned image tag and a restart policy, and mount ~/.cache/vllm and ~/.cache/flashinfer as NVIDIA's troubleshooting page suggests, because the first FlashInfer autotune can exceed vLLM's default one-hour engine timeout (raise VLLM_ENGINE_READY_TIMEOUT_S if it does).

vLLM, NIM or TensorRT-LLM?

vLLM is the documented, licence-free path on the Station today. NVIDIA NIM packages vLLM or SGLang behind OpenAI-compatible endpoints with NVIDIA support, but it needs an NVIDIA AI Enterprise licence, which is not included with the Station, and the product page footnote only says NIM support is "planned". Per-model NIM support for compute capability 10.3 on arm64 could not be confirmed. TensorRT-LLM and SGLang are pre-installed, and NVIDIA publishes an SGLang playbook for the Station; consider them when you need a specific optimisation, and benchmark against vLLM on your own prompts.

Can vLLM run in a MIG slice or across two Stations?

The GB300 supports up to seven MIG instances, with profiles from 1g.31gb to 7g.251gb in NVIDIA's MIG User Guide, and NVIDIA's MIG playbook pins work to a slice with CUDA_VISIBLE_DEVICES=<MIG UUID>. A smaller model can be served from a slice while another team uses the rest; NVIDIA has not published a vLLM-on-MIG recipe for the Station, so test it. MIG layouts do not survive a reboot. Our MIG post covers the sharing model. Two Stations can be cabled back-to-back with NVIDIA's "Connect Two Stations" playbook, over RoCEv2 with no inter-node NVLink; no published multi-node vLLM throughput for two Stations was found.

Buying in Singapore

SourceIT lists DGX Station GB300 systems from six OEMs, supplied through each OEM's channel: the ASUS ExpertCenter Pro ET900N G3, Dell Pro Max with GB300, HP ZGX Fury G1n, MSI XpertStation WS300, GIGABYTE W775-V10-L01 and Supermicro ARS-511GD-NB-LCC. All are price on request; that is a supplier condition rather than a SourceIT choice. Before you order, plan where model files will live using our storage planning worksheet, and see the setup guide for the first session. We can help with the quotation, configuration check and the end-user paperwork suppliers ask for. Use Request for Quotation on the product page, call (65) 6978 3502, email Sales@sourceit.com.sg or use our contact page.

Frequently asked questions

Does vLLM run natively on the DGX Station's Arm CPU? Yes. vLLM documents aarch64 containers for Grace systems, and NVIDIA's Station playbook runs the upstream vllm/vllm-openai image. Pin the exact tag you have tested.

Can vLLM use all 748 GB of memory? Not as one fast pool. vLLM sizes itself against the 252 GB of HBM3e. The 496 GB of LPDDR5X is used only if you enable offload, and single-user speed drops sharply when large parts of a model sit there.

Why does vLLM run out of memory on a Station with an RTX PRO card? With --gpus all it may pick the RTX PRO card instead of the GB300. NVIDIA's playbook selects the GB300 by UUID to avoid this.

Is the vLLM API secure by default? No. NVIDIA warns that binding to 0.0.0.0 can expose an unauthenticated API. Require an API key, restrict the bind address and publish through a gateway.

Do I need NVIDIA AI Enterprise to run vLLM? No. vLLM is open source and pre-installed in the Station's software stack. NVIDIA AI Enterprise is a separate licence, needed for NIM and NVIDIA software support.