Qwen3 Next 80B-A3B in Q4_K_M with an 8k context takes 46.5 GB of GPU memory. The cheapest option right now is RTX A6000 48 GB, from $0.567/h.
The weight file and the total GPU memory with the context cache. One GPU, one request.
| Quantisation | File | 8K context | 32K context | 128K context | Fits on |
|---|---|---|---|---|---|
| BF16 | 148.5 | 149.8 | 150.4 | 152.6 | B200 180 GB |
| FP8 | 76.4 | 77.7 | 78.3 | 80.5 | A800 PCIe 80 GB |
| Q8_0 | 79.0 | 80.3 | 80.8 | 83.1 | H100 NVL 94 GB |
| Q6_K | 61.0 | 62.3 | 62.9 | 65.1 | RTX PRO 5000 72 GB |
| Q5_K_M | 52.9 | 54.2 | 54.8 | 57.0 | RTX PRO 5000 72 GB |
| Q4_K_M | 45.2 | 46.5 | 47.0 | 49.3 | RTX A6000 48 GB |
| Q3_K_M | 35.7 | 37.0 | 37.5 | 39.8 | A100 PCIe 40 GB |
| Q2_K | 27.2 | 28.5 | 29.0 | 31.3 | Tesla V100 Super 32 GB |
How much memory the cache takes per request in FP16. A Q8 cache is about half. 24 KB per token.
GPUs from the catalog the model fits on, by price per hour.
The price is the cheapest GPU of the model × the GPU count. A 2–8 GPU server can cost a little more per GPU.
For an API under load there is the vLLM template with an OpenAI-compatible API.
File sizes checked against Hugging Face on 2026-10-19. 1 GB here is 1024³ bytes, like GPU memory. Hugging Face lists decimal gigabytes, so its numbers are 7% larger.