Home › Blog › AI Infrastructure
AI Infrastructure
15 articles
-
Why Data Locality Matters for AI Performance
Data locality for AI explained: moving data vs moving compute, transfer-time maths for 1GbE to 100GbE, model load times and a tiered storage layout.
Wei Fei
·
-
Vector Database vs Traditional Database: What Changes for AI?
Vector databases vs traditional databases: embeddings, semantic search, when pgvector is enough, memory sizing maths and the hardware to run them.
Wei Fei
·
-
AI Data Pipeline Explained: From Raw Data to Model Deployment
The AI data pipeline explained stage by stage: ingestion, storage, preprocessing, training or indexing, deployment and monitoring, with hardware for each.
Wei Fei
·
-
What Is Confidential Computing for AI? Protecting Data During Processing
Confidential computing for AI explained: protecting data in use with CPU and GPU trusted execution environments, attestation, limits and hardware.
Wei Fei
·
-
What Is CXL Memory? Could It Change Future AI Server Architecture?
CXL memory explained: memory expansion and pooling over PCIe, CXL 1.1 to 4.0, what it can and can't do for AI servers, and what to buy today.
Wei Fei
·
-
What Is Disaggregated AI Inference? Separating Prefill and Decode
Disaggregated inference explained: prefill vs decode, KV cache transfer, NVIDIA Dynamo, when splitting phases across GPUs pays off and when it doesn't.
Wei Fei
·
-
Edge AI vs Cloud AI: Latency, Connectivity and Where to Run Your AI
Edge AI runs the model where the data is produced; cloud AI sends the data away. How latency, bandwidth, connectivity and privacy decide between them.
Wei Fei
·
-
AI Model Monitoring and Observability: What to Watch Once AI Is in Production
Monitoring tells you an AI service is unhealthy; observability tells you why. The four layers of signals to track and a starter set of alerts.
Wei Fei
·
-
Why Your AI Server Needs a NAS: Synology Storage for AI Workloads
An AI server's internal SSD is for speed, not safekeeping. What a NAS adds: datasets, model libraries, snapshots and backup, and what it cannot do.
Wei Fei
·
-
GPU Partitioning Explained: NVIDIA MIG vs Virtual GPU
NVIDIA MIG and vGPU both share one GPU between several users. MIG partitions the hardware; vGPU shares it across virtual machines. How to choose.
Wei Fei
·
-
AI Model Serving Explained: From Notebook to Production
AI model serving explained: what changes between a notebook demo and a production service, the layers of a serving stack, key metrics and hardware.
Wei Fei
·
-
TensorRT-LLM vs vLLM: Which Inference Engine Should You Use?
TensorRT-LLM vs vLLM compared: performance, model support, hardware lock-in and operational effort, with a way to benchmark both on your own workload.
Wei Fei
·
-
CUDA vs ROCm vs OpenVINO: Choosing an AI Compute Ecosystem
CUDA vs ROCm vs OpenVINO for developers: hardware support, framework compatibility, Windows and Linux status, and how to choose without lock-in.
Wei Fei
·
-
Docker vs Kubernetes for AI Deployment: When Do Businesses Need Orchestration?
Docker vs Kubernetes for AI: what each does, why GPUs change the picture, and a practical rule for when a business needs container orchestration.
Wei Fei
·
-
NVIDIA NIM vs Ollama: Enterprise AI Deployment Compared
NVIDIA NIM vs Ollama compared: what each is, licensing and cost, performance, support and security, and when a business should move from one to the other.
Wei Fei
·
Wei Fei
