Gemma 3 27B in Q4_K_M with an 8k context takes 17.7 GB of GPU memory. The cheapest option right now is 2× RTX 3060 12 GB, from $0.168/h.
The weight file and the total GPU memory with the context cache. One GPU, one request.
| Quantisation | File | 8K context | 32K context | 128K context | Fits on |
|---|---|---|---|---|---|
| BF16 | 50.3 | 52.6 | 54.5 | 62.0 | RTX PRO 5000 72 GB |
| Q8_0 | 26.7 | 29.0 | 30.9 | 38.4 | Tesla V100 Super 32 GB |
| Q6_K | 20.6 | 22.9 | 24.8 | 32.3 | RTX 3090 24 GB |
| Q5_K_M | 17.9 | 20.2 | 22.1 | 29.6 | RTX 3090 24 GB |
| Q4_K_M | 15.4 | 17.7 | 19.6 | 27.1 | RTX 4000 Ada 20 GB |
| Q3_K_M | 12.5 | 14.8 | 16.7 | 24.2 | RTX 4070 Ti Super 16 GB |
| Q2_K | 9.8 | 12.1 | 13.9 | 21.4 | RTX 4070 Ti Super 16 GB |
How much memory the cache takes per request in FP16. A Q8 cache is about half. 80 KB per token + 1024 × 416 KB.
GPUs from the catalog the model fits on, by price per hour.
The price is the cheapest GPU of the model × the GPU count. A 2–8 GPU server can cost a little more per GPU.
In the Ollama template the model is one command away:
ollama run gemma3:27bFor an API under load there is the vLLM template with an OpenAI-compatible API.
File sizes checked against Hugging Face on 2026-10-19. 1 GB here is 1024³ bytes, like GPU memory. Hugging Face lists decimal gigabytes, so its numbers are 7% larger.