Which GPU do you need for AI? Pick by task and VRAM

Pick a GPU for AI by its memory: the model has to fit in the card's VRAM in full. A chat with a 14B model runs on 12 GB; 27–32B models, FLUX and Wan 2.2 video on 24 GB; Llama 3.3 70B on 48 GB; GPT-OSS 120B on 80 GB. Below: a task-to-memory table, which cards belong to each memory size, and what an hour of each costs to rent right now.

Why memory comes first

A model runs fast only while its weights sit in GPU memory. When memory runs short, part of the model moves to system RAM and generation slows down many times over. So look at the gigabytes first and at the card's generation and speed second.

Speed matters too, but it decides how fast a model answers. Memory decides whether it runs at all. The RTX 3090 and RTX 4090 are different generations, but both have 24 GB and run the same models.

How much memory you need depends on three things: the model's size (its parameter count), quantisation — weight compression (Q4 stores about 4.8 bits per parameter instead of 16) — and the context length, how much text the model keeps in mind. The maths is in How much VRAM an LLM needs.

How much VRAM does each task need?

Chat and code need 12 GB, FLUX images 16 GB, strong models and video 24 GB. The figures come from the real model files.

Which GPUs are good for AI?

The workhorse size is 24 GB: most open models for text, code, images and video run on it. Below are the memory steps, the cards that have them and the tasks each step is the minimum for.

Check the memory for your model

Pick a model, quantisation and context length: the calculator shows the memory it needs and the cheapest catalog cards that hold it.

Gaming card or data-centre card?

To run ready models up to 32B parameters, 24–32 GB gaming cards are enough: the RTX 3090, RTX 4090 and RTX 5090. Data-centre A100, H100 and H200 cards are for models that do not fit in 32 GB, for a single 80 GB+ card, and for fine-tuning.

• LoRA fine-tuning of a 7–8B model: about 24 GB, QLoRA 16 GB. More in choosing a GPU for LoRA.
• Full fine-tuning of a 7B model takes about 112 GB: two 80 GB cards.
• An API for many users on vLLM: every concurrent request keeps its own context cache, so leave plenty of headroom.

Buy a GPU or rent one?

Your own card pays off when it is busy nearly all the time. If you need AI models a few hours a week or are still choosing a model, renting costs less: you pay for the minutes you use and can move to a bigger card at any time. The break-even maths is in Buy an H100 or rent one.

How to run a model on a rented GPU

1. 1

   Work out the memory

   Find your model in the VRAM calculator or the table above. Leave headroom: above 92% of the card, a long conversation may not fit.

2. 2

   Pick a card

   Filter the catalog by memory. The hourly price is shown up front; billing is per minute.

3. 3

   Pick a template

   For a chat, Open WebUI + Ollama or a ready model chat; for an API with several users, vLLM; for images, the ComfyUI and Forge templates. All of them are on the templates page.

4. 4

   Work, then stop the server

   The server starts in 2–5 minutes. When you are done, stop it: the GPU charge stops and only the disk is billed.

What can go wrong

• A tight fit. The model starts, but a long context or a large image no longer fits. Pick a card where the model takes at most 90% of the memory.
• The wrong quantisation. The same model in BF16 is 3–4 times larger than in Q4_K_M. Check which file you download.
• AMD and Mac. Most AI tools — PyTorch, ComfyUI, vLLM — are built first for NVIDIA GPUs with CUDA. On other platforms some models are harder or slower to run.

How we counted

Language models: weights are the real GGUF and native file sizes on Hugging Face, the context cache follows each model's architecture, plus about 0.75 GB of CUDA runtime; 8K context, one request. Images and video: the model, VAE and working memory of a ComfyUI generation. A card fits when the model takes at most 97.5% of its memory. 1 GB is 1024³ bytes. Data as of 2026-10-19; a table for 35 language models is in How much VRAM do open LLMs really need?.