How much VRAM does Llama 3.3 70B need?

In Q4_K_M with an 8k context, 43.1 GB: weights 39.6 GB, context cache 2.5 GB and about 1 GB of runtime memory. In BF16, 134.9 GB.

Does Llama 3.3 70B run on an RTX 4090 with 24 GB?

It does not fit on one RTX 4090 even in Q4_K_M: it needs 43.1 GB. A 48 GB GB card or several RTX 4090s will do.

How much does it cost to run Llama 3.3 70B?

The cheapest fitting server right now is 4× RTX 3060 12 GB, from $0.348/h. Billing is per minute: stop the server and only its disk is billed.

Which quantisation should I pick?

Q4_K_M is the chat default: close to the original quality at a third of BF16's size. Q5_K_M and Q6_K are more accurate, Q8_0 is almost the original. For vLLM use FP8 or BF16.

Llama 3.3 70B: how much VRAM it needs and which GPU runs it

Llama 3.3 70B in Q4_K_M with an 8k context takes 43.1 GB of GPU memory. The cheapest option right now is 4× RTX 3060 12 GB, from $0.348/h.

Find a server Open in the calculator

VRAM by quantisation

The weight file and the total GPU memory with the context cache. One GPU, one request.

Quantisation	File	8K context	32K context	128K context	Fits on
BF16	131.4	134.9	142.4	172.4	H200 NVL 141 GB
Q8_0	69.8	73.3	80.8	110.8	A800 PCIe 80 GB
Q6_K	53.9	57.4	64.9	94.9	RTX PRO 5000 72 GB
Q5_K_M	46.5	50.0	57.5	87.5	RTX PRO 5000 72 GB
Q4_K_M	39.6	43.1

File sizes checked against Hugging Face on 2026-10-19. 1 GB here is 1024³ bytes, like GPU memory. Hugging Face lists decimal gigabytes, so its numbers are 7% larger.

Llama 3.3 70B: how much VRAM it needs and which GPU runs it

VRAM by quantisation

Llama 3.3 70B: how much VRAM it needs and which GPU runs it

VRAM by quantisation

Context cache

Cheapest GPUs that run it

How to run it on gpu.nz

Questions