Running DeepSeek V4 Flash yourself: how many GPUs you need

DeepSeek V4 Flash at Q4 needs 145.5 GB of GPU memory (8K context): that is 2 H200 cards (141 GB each) or 4 cards of 80 GB with headroom. One card can do it if it is big enough: 180 GB for Q4, 141 GB for Q3 (121.5 GB) and 96 GB for Q2 (91.3 GB), a tight fit. Below: a quantisation-by-card table, the cost of a long context, a calculator and how to run it with vLLM and llama.cpp.

What is DeepSeek V4 Flash and how does it differ from R1 and V3.2?

DeepSeek V4 Flash is an open model from DeepSeek, the smaller one in the V4 family (the larger is called V4-Pro). Its model card says the licence is MIT. The Hugging Face counter shows 291B parameters and the context goes up to 1M tokens.

It is a Mixture-of-Experts model: about 13B parameters work on each token. But every expert has to sit in GPU memory. So the active parameters usually speed up the answer, while the number of cards you need stays the same.

For a GPU, the main difference is size. V4 Flash is 2.4 times smaller by parameter count, 2.6 times lighter than V3.2 and R1 at Q4, and its context cache per token is 5.6 times smaller. So it fits 2–4 cards, while V3.2 and R1 need 4–8 at Q4. Those two models, and the R1 distills for a single small card, are covered in How to run DeepSeek locally or on your own server.

How much VRAM does DeepSeek V4 Flash need?

145.5 GB at Q4, 121.5 GB at Q3 and 91.3 GB at Q2, with an 8K context and one request. Quantisation compresses the weights: fewer bits per weight mean less memory and, usually, a bigger loss in quality. The GGUF files here come from unsloth: UD-Q4_K_XL (the calculator lists it in the Q4_K_M slot), Q3_K_M and UD-Q2_K_XL. The general rule of thumb is in How much VRAM an LLM needs.

The original FP4/FP8 weights take 149.7 GB: close to Q4 in memory, without the loss from compression. vLLM runs them, and llama.cpp runs the GGUF files. Sizes for every version are on the DeepSeek V4 Flash page.

How many GPUs: quantisation against card size

Weights and cache are split evenly between the cards (tensor parallelism), and each card keeps its own CUDA runtime memory. We take the smallest number of cards out of 1, 2, 4 and 8 that holds the model. A cell shows how many cards of that size you need; a dash means even eight are not enough.

“Tight” means more than 92% of the card's memory is used: the model starts, but a long request may not fit. What the table shows:

• Q4: 2 H200 cards (53% of each card). On 80 GB cards: 4 cards with headroom or 2 cards tightly.
• Q3: 1 H200 card (88% of memory) or 2 cards of 80 GB.
• Q2: 1 card of 96 GB, 98% of memory used: a short context and one request.
• 24 and 48 GB cards: Q4 fits 8 and 4 of them. But the more cards, the more the speed depends on the links between them. We did not measure it.

This is a memory calculation. The vLLM recipe for V4 Flash (recipes.vllm.ai) describes layouts of 4–8 cards for H200, B200, B300 and RTX PRO 6000. For 8 × H100 it has only community reports, which the recipe does not confirm. We have not run a two-card setup.

What a compressed cache buys you: up to 1M tokens of context

On most models the context cache grows fast: going from 8K to 128K adds 30.0 GB to the dense R1 Distill Qwen 32B. V4 Flash has a compressed cache: the calculator estimates about 12.5 KB per token. A 128K context adds 1.4 GB, and the full million adds 12.1 GB.

That figure is an estimate from the architecture, not a measurement. DeepSeek's model card states a cache saving only for the larger V4-Pro: about 10% of the V3.2 cache at a million tokens. There is no such figure for Flash. The same card describes a hybrid attention of compressed sparse (CSA) and heavily compressed (HCA) layers.

DeepSeek recommends running the Think Max mode (maximum reasoning effort) with a context of at least 384K tokens. For V4 Flash at Q4 that is 150.0 GB: only 4.5 GB more than at 8K.

A million tokens at Q4 take 157.6 GB. That is still 2 H200 cards or 4 cards of 80 GB.

Check your own setup

Pick the model, quantisation and context length: the calculator shows the memory and the catalog servers with enough cards, the cheapest first. Its context goes up to 128K; the million is in the table above.

Is one GPU enough?

Only if the card is very large. Q4 fits one 180 GB card, Q3 fits 141 GB (H200), Q2 fits 96 GB. Cards of 80 GB and smaller do not hold even Q2.

Q2 on a 96 GB card is a corner case. It still holds a 128K context (92.7 GB) but not a million: that needs a 141 GB card. We did not measure Q2 quality, so test it on your own tasks.

Look for servers with several cards of one type in the catalog, as availability changes. If there is none, you are left with Q2 on one card and a short context, or another model. For one small card, the R1 distills fit: R1 Distill Qwen 32B at Q4 takes 21.5 GB (24 GB card) and R1 0528 Qwen3 8B takes 6.8 GB (8 GB card). They are not V4: they are Qwen models trained on R1's reasoning.

What does an hour cost?

The price of an hour is the number of cards times the price of one. For Q4 that is 2 H200 cards or 4 cards of 80 GB. Prices change every minute, so live cards are below. A server with several cards may cost a little more per card; the calculator above shows the exact price.

How to run DeepSeek V4 Flash on a rented server

We have no ready template for this model, so the setup is manual. The official model card on Hugging Face gives commands for vLLM and SGLang, and unsloth publishes the GGUF files for llama.cpp. vLLM suits an API with parallel requests; llama.cpp suits the Q2–Q4 files and a single user. V4 support in both projects is new and changes fast, so take recent releases.

1. 1

   Pick a multi-card server and the vLLM template

   In the catalog find a server with 4–8 cards (H200 or 80 GB). 2 H200 cards would hold it by memory, but the vLLM recipe describes layouts of 4 and 8 cards. Press Rent, choose the vLLM template and a disk of at least 240 GB: the files weigh about 160 GB.

2. 2

   Connect over SSH

   The connection command is on the server page. The template first starts its own default model; we replace it now.

3. 3

   Replace the model with DeepSeek V4 Flash

   Put the number of cards in --tensor-parallel-size and the rental token in --api-key. The flags come from the vLLM recipe. The repository has no chat template, which is why --tokenizer-mode deepseek_v4 is needed.

   pkill -f api_server
   nohup vllm serve deepseek-ai/DeepSeek-V4-Flash --host 0.0.0.0 --port 8000 \
     --api-key <rental token> --tensor-parallel-size <number of cards> --enable-expert-parallel \
     --kv-cache-dtype fp8 --block-size 256 --trust-remote-code \
     --tokenizer-mode deepseek_v4 --reasoning-parser deepseek_v4 \
     --max-model-len 131072 > /var/log/vllm.log 2>&1 &
   tail -f /var/log/vllm.log

4. 4

   Wait for the download

   The model downloads and compiles, which takes longer than for small models. The server is ready when the log says the API has started.

5. 5

   Send a request

   The API address is the «API (port 8000)» button on the server page. The model name in the request matches the vllm serve argument.

   curl https://<id>-8000.run.gpu.nz/v1/chat/completions \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{"model": "deepseek-ai/DeepSeek-V4-Flash", "messages": [{"role": "user", "content": "Hello"}]}'

6. 6

   Stop the server when you are done

   Billing is per minute: a stopped server costs nothing for the GPUs, only its disk.

For one user or fewer cards, build llama.cpp on the Ubuntu + CUDA template and take a GGUF file. llama.cpp splits the layers across all visible cards by itself:

apt-get update && apt-get install -y build-essential cmake git curl libcurl4-openssl-dev
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --target llama-server
./llama.cpp/build/bin/llama-server -hf unsloth/DeepSeek-V4-Flash-GGUF:UD-Q4_K_XL \
  --port 8080 -ngl 99 --ctx-size 32768

The Ubuntu template has no web addresses, so forward the port over SSH from your computer and open http://localhost:8080:

ssh -p <port> -L 8080:localhost:8080 root@<address>

What can go wrong

• The vLLM version. The recipe asks for vLLM 0.20.0 or newer. Take a release, not the main branch: LMCache's guide says V4 Flash failed to start on it. Check the version in the template with python3 -c "import vllm; print(vllm.__version__)".
• A newer revision. DeepSeek-V4-Flash-0731 is out too, and the Hugging Face counter shows 304B parameters for it. Our sizes are for the main version, so leave extra memory for 0731.
• Parallel requests multiply the cache. The figures above are for one request. In vLLM every concurrent request keeps its own cache, and vLLM takes up to 90% of each card by default. For an API, choose setups without the “tight” mark.
• We did not measure speed. Every figure is a memory calculation. Answer speed depends on the number of cards and the links between them, so test it on your own server before you build the model into a service.

How we counted

Weights are the real file sizes on Hugging Face: GGUF from unsloth/DeepSeek-V4-Flash-GGUF and the FP4/FP8 original from deepseek-ai/DeepSeek-V4-Flash. The V4 Flash context cache is the calculator's estimate, 12.5 KB per token; for V3.2 and R1 it is 70.3 KB from config.json. The runtime is about 0.75 GB plus the logits buffer on each card. With several GPUs, weights and cache split evenly.

1 GB here is 1024³ bytes, as on GPUs, while Hugging Face shows decimal gigabytes: 160 GB instead of 148.7. Parameter counts come from the Hugging Face counter (the V4 Flash card says 284B; for V3.2 and R1 the counter includes the auxiliary MTP module, hence 685B instead of the 671B in their names). The active parameters, the licence and the Think Max advice come from the model card, and the commands from the same card, the vLLM recipe and the unsloth documentation. Data as of 2026-10-19; the method is in the calculator, and one table for 35 models is in How much VRAM do open LLMs really need?.