Self-Host DeepSeek V4 with vLLM: 2026 Deployment Guide
In this article
- Should You Self-Host DeepSeek V4?
- Hardware Requirements
- V4-Flash
- V4-Pro
- Downloading Model Weights
- vLLM Deployment Setup
- Serve V4-Flash on 4× A100 80GB
- Serve V4-Flash on 2× H200
- Serving DeepSeek V4.1 Flash on vLLM
- Serve V4.1 Flash on 4x H200
- Serve V4.1 Flash on 8x H100 80GB
- Thinking is on by default
- Expert Parallelism and Tensor Parallelism
- Quantization Options
- Configuring the 1M Context Window
- AWS Instance Selection
- Break-Even Analysis: Self-Host vs DeepSeek API
- What to Read Next
Should You Self-Host DeepSeek V4?
DeepSeek V4 ships under the MIT license with open weights on Hugging Face — V4 Flash is a 284B-parameter sparse MoE with 13B active parameters in the original preview and a mixed FP4/FP8 checkpoint of roughly 158GB. V4 Pro is substantially larger at about 1.6T parameters and roughly 862GB of weights. The license permits commercial use, fine-tuning, and infrastructure modification.
September 2026 status: The commands below use DeepSeek's official Hugging Face repositories and vLLM's current
vllm serveentry point. DeepSeek's hosted list price is now $0.14/MTok cache-miss input and $0.28/MTok output for Flash, and $0.435/$0.87 for Pro. Recalculate the economics from the official pricing page before reserving hardware.
Deploying V4.1 Flash? It is a different, much larger checkpoint (510.3 GB) with its own vLLM flags. Jump to Serving DeepSeek V4.1 Flash on vLLM. For one machine with mixed GPU and system memory, see Run DeepSeek V4.1 Flash Locally with llama.cpp.
Three legitimate reasons to self-host:
- Data sovereignty — DeepSeek's hosted API routes through infrastructure outside your control. For regulated industries, defense-adjacent work, or GDPR-sensitive applications, keeping inference on your own hardware removes that dependency entirely.
- Fine-tuning — The MIT license lets you modify and serve fine-tuned variants. The hosted API does not.
- Consistent latency — Self-hosted inference has no quota limits or API rate fluctuations.
The reason not to self-host is usually cost. At DeepSeek's own API pricing ($0.14/M input, $0.28/M output), break-even against AWS on-demand pricing for V4-Flash requires roughly 3–4 billion tokens/day of sustained throughput on a reserved instance. Most teams never reach that volume. Run your own numbers before committing.
Hardware Requirements
V4-Flash
V4-Flash is the practical self-hosting target. The FP4+FP8 Instruct checkpoint weights are ~158GB. Add ~10GB for a full 1M-token KV cache (DeepSeek V4 uses only ~7% of V3.2's KV cache footprint) and a few GB of runtime overhead: total VRAM budget is roughly 170–175GB.
| GPU Config | Total VRAM | Fit V4-Flash? | Notes |
|---|---|---|---|
| 2× H200 141GB | 282 GB | ✅ Comfortable | Recommended; headroom for full 1M context |
| 2× RTX Pro 6000 Blackwell | 192 GB | ✅ Comfortable | More affordable than H200; NVLink required |
| 4× A100 80GB | 320 GB | ✅ Works | vLLM prefers power-of-2 TP; 2× A100 (160GB) is just under budget |
| 2× A100 80GB | 160 GB | ⚠️ Tight | Below budget once KV cache is loaded at full context; reduce max-model-len |
| 4× RTX 4090 | 96 GB | ❌ No | Only feasible with INT4 quantization (~80GB); expect quality degradation |
The 4× A100 recommendation in many guides is a vLLM artifact: tensor parallelism works best with power-of-two GPU counts. 2× A100 gives 160GB which is technically below the full-context budget, so the next safe power-of-two is 4× (320GB). The extra headroom is a side effect, not a requirement.
V4-Pro
V4-Pro at ~862GB is a real cluster problem:
| Config | Total VRAM | Notes |
|---|---|---|
| 8× H200 141GB (single node) | 1,128 GB | Minimum single-node config |
| DGX H200 | 1,128 GB | Purpose-built; NVLink fabric included |
| 2× p5.48xlarge (16× H100 80GB) | 1,280 GB | Multi-node; requires NVLink + InfiniBand |
Unless you specifically need V4-Pro's superior agentic coding and knowledge depth over V4-Flash, V4-Flash delivers 85–95% of V4-Pro quality at a fraction of the infrastructure cost.
System RAM: 256GB+ for V4-Flash. Storage: 500GB NVMe minimum.
Downloading Model Weights
Both models live under the deepseek-ai organization on Hugging Face. Use the Instruct FP4+FP8 checkpoints for production:
pip install huggingface_hub
# V4-Flash (~158GB)
huggingface-cli download deepseek-ai/DeepSeek-V4-Flash \
--local-dir ./deepseek-v4-flash
# V4-Pro (~862GB)
huggingface-cli download deepseek-ai/DeepSeek-V4-Pro \
--local-dir ./deepseek-v4-pro
For faster transfers, enable hf_transfer:
pip install hf_transfer
export HF_HUB_ENABLE_HF_TRANSFER=1
With a good network connection and HF token set, this can reach 1–2 GB/s on capable infrastructure.
vLLM Deployment Setup
Use the current vLLM release rather than pinning this guide to a historical minimum. DeepSeek's official model card provides vLLM serving instructions for the custom architecture; reproduce the exact package and model revisions in your deployment lockfile after validation.
pip install --upgrade vllm
Serve V4-Flash on 4× A100 80GB
vllm serve ./deepseek-v4-flash \
--tensor-parallel-size 4 \
--max-model-len 131072 \
--trust-remote-code \
--port 8000
Serve V4-Flash on 2× H200
vllm serve ./deepseek-v4-flash \
--tensor-parallel-size 2 \
--max-model-len 1048576 \
--trust-remote-code \
--port 8000
vLLM exposes an OpenAI-compatible API. Point any OpenAI SDK client at it by changing base_url:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="token")
response = client.chat.completions.create(
model="deepseek-v4-flash",
messages=[{"role": "user", "content": "Explain mixture-of-experts routing."}],
)
print(response.choices[0].message.content)
Serving DeepSeek V4.1 Flash on vLLM
Added 22 November 2026. DeepSeek V4.1 Flash is not a drop-in replacement for V4 Flash. Per DeepSeek's model card, it has a 552B-parameter backbone in a causal encoder-decoder layout, activating 8B parameters per token during prefill and 16B during decode, plus 196B parameters of Engram conditional memory. The official checkpoint is 510.3 GB, more than three times V4 Flash's 158GB. The commands below are the generated output of the official vLLM recipe (last updated 17 October 2026), reproduced unchanged. We have not run them ourselves.
Requirements that differ from V4 Flash:
- vLLM 0.31.0 or later. The architecture first shipped in 0.30.0, but the recipe sets 0.31.0 as its minimum.
--tokenizer-mode deepseek_v41encodes prompts in Python, which is why DeepSeek's release can ship without a Jinja chat template.--engram-config '{"cpu_offload":true}'moves the Engram tables to pinned host memory and reads them through UVA. The recipe states the output is unchanged. It is what makes 4x H200 practical.VLLM_ENGINE_READY_TIMEOUT_S=3600: startup is slow enough that the default readiness timeout can expire.
The recipe's NVIDIA configurations, all serving the 1017 checkpoint:
| Hardware | Parallelism | What the recipe adds |
|---|---|---|
| 8x H100 80GB | TP8 | Engram offload, --max-num-batched-tokens 4096, V2 model runner. The smallest NVIDIA node that serves it. |
| 4x H200 141GB | TP4 | Engram offload. Measured: 81.2 GiB of resident weights and 38.5 GiB of KV per GPU (about 20M tokens). |
| 2x B200 / B300 / GB200 / GB300 | TP2 | Engram offload, FlashInfer sparse MLA attention, FP8 KV cache |
| 1x DGX Station GB300 | TP1 | Engram tables plus 60 GiB of routed experts in Grace memory, 244 GB GPU budget |
Serve V4.1 Flash on 4x H200
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
export VLLM_ENGINE_READY_TIMEOUT_S=3600
export VLLM_USE_RUST_FRONTEND=1
vllm serve deepseek-ai/DeepSeek-V4.1-Flash-1017 --tokenizer-mode deepseek_v41 --tensor-parallel-size 4 --engram-config '{"cpu_offload":true}' --tool-call-parser deepseek_v41 --enable-auto-tool-choice --reasoning-parser deepseek_v41 --mm-encoder-tp-mode data
Serve V4.1 Flash on 8x H100 80GB
export VLLM_ENGINE_READY_TIMEOUT_S=3600
export VLLM_USE_V2_MODEL_RUNNER=1
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
export VLLM_USE_RUST_FRONTEND=1
vllm serve deepseek-ai/DeepSeek-V4.1-Flash-1017 --tokenizer-mode deepseek_v41 --engram-config '{"cpu_offload":true}' --max-num-batched-tokens 4096 --tensor-parallel-size 8 --tool-call-parser deepseek_v41 --enable-auto-tool-choice --reasoning-parser deepseek_v41 --mm-encoder-tp-mode data
The recipe explains the 4096 cap: the sparse-attention indexer allocates a logits buffer sized by batched tokens times context length, and at 8192 x 1M tokens that buffer alone is 16 GiB, which does not fit beside the weights on 80GB cards.
For Blackwell and AMD configurations, generate the exact command from the recipe page: their attention backends and scheduler limits differ per SKU.
Thinking is on by default
The recipe warns that with no reasoning options set, thinking is on at effort 50. A request with a small max_tokens can spend its whole budget on the reasoning trace and come back with empty content and finish_reason=length. That looks like a broken model, but it isn't. Set the mode explicitly:
resp = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V4.1-Flash-1017",
messages=[{"role": "user", "content": "What is 17*19?"}],
extra_body={"chat_template_kwargs": {"thinking": True, "reasoning_effort": 25}},
)
The effort scale differs between DeepSeek's API and vLLM. In vLLM, low, high, xhigh and max map to 25, 50, 75 and 100; DeepSeek's API maps low, high and max to 50, 75 and 100. A prompt tuned at high on the API therefore runs at effort 50 on vLLM, so pass an integer when you need parity.
The recipe's verification badges and measurements were taken on the September launch checkpoint. The 1017 weights share its config and memory footprint but were not separately re-run.
Expert Parallelism and Tensor Parallelism
MoE models benefit from two distinct parallelism strategies, and understanding the difference determines how efficiently you use multi-GPU setups:
Tensor Parallelism (TP): Splits individual layer weight matrices across GPUs. Required when a single layer is too large for one GPU. Set with --tensor-parallel-size. Communication-heavy — works best on GPUs connected by NVLink.
Expert Parallelism (EP): Routes different MoE expert sub-networks to different GPUs. Because MoE only activates a small fraction of experts per token, EP lets you spread experts without duplicating all parameters everywhere. Lower communication overhead than TP.
For V4 Flash on two or four GPUs, start with tensor parallelism because the configuration is explicit and easy to reproduce. Expert parallelism is not something to assume: enable it deliberately with --enable-expert-parallel, then compare throughput, interconnect traffic, and tail latency under the same load before keeping it.
For V4 Pro on eight or more GPUs, combine TP and EP only after measuring the topology. Record the tensor-parallel size, data-parallel size, expert-parallel setting, vLLM revision, and NCCL environment alongside every benchmark so a faster result is reproducible.
Quantization Options
DeepSeek ships V4 in two formats on Hugging Face:
- FP8 Mixed (Base checkpoints): Higher quality, larger footprint. Dense parameters in FP8.
- FP4+FP8 Mixed (Instruct checkpoints): MoE expert weights in FP4, other parameters in FP8. Recommended format — balances quality and memory.
Community quantizations (GGUF, AWQ, GPTQ) are available and can compress V4-Flash to ~80GB at INT4, potentially fitting on 4× RTX 4090. The trade-off is measurable quality degradation on reasoning, math, and complex instruction-following tasks.
Practical guidance: The official FP4+FP8 Instruct checkpoint is already aggressively quantized from the full FP32 baseline. For production inference, stick with the official format unless VRAM constraints leave no alternative. Reserve INT4 community quantizations for development or evaluation on consumer hardware.
Configuring the 1M Context Window
V4's hybrid CSA+HCA attention architecture dramatically reduces KV cache memory compared to V3.2 — roughly 7% of the footprint at equivalent context length. A full 1M-token context on V4-Flash consumes approximately:
- 158GB weights
- ~10GB KV cache at full 1M context
- ~5–7GB runtime overhead
Total: ~170–175GB
Set --max-model-len in vLLM to match your VRAM headroom:
# Start conservative on 4x A100 (320GB total, ~150GB headroom after weights)
--max-model-len 131072 # 128K
# Full 1M on 2x H200 (282GB total, ~110GB headroom after weights)
--max-model-len 1048576 # 1M
Monitor GPU memory during inference under load and adjust upward until you're within 5–10GB of VRAM capacity.
AWS Instance Selection
| Instance | GPUs | Total VRAM | On-Demand ($/hr) | Use Case |
|---|---|---|---|---|
| p5.48xlarge | 8× H100 80GB | 640 GB | ~$55 | V4-Flash (comfortable) |
| p5e.48xlarge | 8× H200 141GB | 1,128 GB | ~$40–50 | V4-Pro (single node) |
| p5en.48xlarge | 8× H200 141GB (200Gbps) | 1,128 GB | ~$63 | V4-Pro + faster fabric |
| 2× p5.48xlarge | 16× H100 80GB | 1,280 GB | ~$110 | V4-Pro (multi-node) |
1-year reserved instances cut costs roughly 40%. Spot instances are viable for batch inference but not for latency-sensitive serving due to interruption risk.
For most teams evaluating self-hosting, a single p5.48xlarge running V4-Flash is the practical starting point.
Break-Even Analysis: Self-Host vs DeepSeek API
DeepSeek's API pricing (verified 14 September 2026):
- Input: $0.14 per million tokens
- Output: $0.28 per million tokens
- Blended 50/50 rate: ~$0.21 per million tokens
At $0.21/M, spending $790/day (reserved p5.48xlarge) equates to roughly 3.8 billion tokens/day. A single 8× H100 node running V4-Flash cannot physically sustain that throughput.
The cost argument almost never closes. Self-host when:
- Data residency or regulatory compliance require on-premises or VPC inference
- You need custom fine-tuned weights the hosted API cannot serve
- Your workload requires consistent low-latency that API quotas cannot guarantee
For all other cases, the API is cheaper and requires no infrastructure maintenance.
What to Read Next
- Deploy Models on RunPod — rent GPU infrastructure by the hour for V4-Flash evaluation before committing to reserved instances
- Run DeepSeek V4 Flash Locally with llama.cpp — single-GPU local setup using a modified llama.cpp build and GGUF quantization
- Production-Grade RAG Architecture — connect your self-hosted V4 endpoint to a retrieval pipeline
Frequently asked questions
How many GPUs are required to self-host DeepSeek V4 Flash?
The official mixed FP4/FP8 checkpoint is roughly 158GB before KV cache and runtime overhead. Two 141GB H200 GPUs or another configuration with at least about 175GB of usable aggregate VRAM is the practical starting point for long-context serving.
What vLLM command serves DeepSeek V4 Flash?
Use `vllm serve deepseek-ai/DeepSeek-V4-Flash` with a tensor-parallel size matching the GPUs assigned to the process. Start with a conservative `--max-model-len`, run load tests, and only then increase context length within measured KV-cache headroom.
Is self-hosting DeepSeek V4 cheaper than the official API?
Usually not at ordinary utilization. At the September 2026 Flash list prices of $0.14 per million cache-miss input tokens and $0.28 per million output tokens, an expensive multi-GPU node needs extremely high sustained token volume before hardware rental becomes cheaper.
How many GPUs does DeepSeek V4.1 Flash need on vLLM?
Far more than V4 Flash: the official V4.1 Flash checkpoint is 510.3 GB and vLLM's recipe lists a 614 GB minimum without offload. With the Engram tables moved to host memory, the recipe's NVIDIA configurations are 8x H100 80GB (TP8), 4x H200 (TP4), 2x B200/B300/GB200/GB300 (TP2), or one DGX Station GB300 that keeps experts and Engram tables in Grace memory. It requires vLLM 0.31.0 or later.
Should I deploy DeepSeek V4 Flash or V4 Pro?
Flash is the practical self-hosting target because the checkpoint is far smaller. Pro requires a large multi-GPU server or cluster, so use it only when measured workload quality justifies the additional memory, networking, and operational cost.
Related Guides
Run DeepSeek V4.1 Flash Locally with llama.cpp (v1.0.0)
Run DeepSeek V4.1 Flash 1017 locally with llama.cpp v1.0.0 and the official ggml-org GGUF: memory sizing from the real file layout, the exact server flags, and workarounds for each open bug.
Self-Hosting Your First LLM: Complete Hardware, Benchmark & Deployment Playbook
A complete engineering playbook for self-hosting production LLMs for AI agents: benchmark selection, quantization trade-offs, cloud GPU instance economics, and zero-switch proxy architecture.
How to Run DeepSeek R1 Locally with Ollama: Full Setup Guide
Install DeepSeek R1 locally using Ollama in under 5 minutes. Covers model variant selection from 1.5B to 671B, visible chain-of-thought reasoning, REST API usage, Python integration, and building a simple RAG application.