Best Ollama models: what to run on 8, 16, 24, 48 and 80 GB

Choose a model by its video memory, because memory decides which model fits on the card in full. Below is a selection for five steps: 8, 16, 24, 48 and 80 GB. Every model installs with one command, ollama pull. On gpu.nz you can download a model from the Open WebUI + Ollama template, or enter its name in the «Model» field at launch, and it downloads on the first start.

How to read the table

The sizes are the files of Ollama library tags. A tag's quantisation is written in its description, and the file size shows how strongly the weights are compressed. The real memory use is about 20% higher because of the context and buffers, and a long conversation adds KV-cache memory on top. Leave headroom on the card. For the full calculation, see our guide how much video memory an LLM needs.

Two rows are mixture-of-experts (MoE) models: qwen3:30b and mixtral:8x7b. All of their parameters sit in memory, but only part of them work for each token, so generation is faster than in a dense model of the same size. The memory still has to hold the whole model.

Anything above 80 GB in file size needs two cards or a server with several GPUs. For example, qwen3:235b weighs about 142 GB, so it needs a multi-card setup.

Model families at a glance

• Qwen3 is a general line with strong multilingual text and a switchable reasoning mode. Sizes run from compact variants up to 235B.
• Gemma 3 and Gemma 4 are Google's models. Gemma 3 variants from 4B up, and every Gemma 4 variant, accept images.
• Llama 3.1 and 3.3 cover English text, instructions and tool calls.
• The distilled DeepSeek R1 versions are models for step-by-step reasoning. For details, see the guide how to run DeepSeek locally.
• gpt-oss is OpenAI's open-weight line: the 20B model fits in 16 GB, and the 120B model fits one 80 GB card.
• Qwen2.5-Coder is built for code: completion, explanation and edits.
• Mistral Small and Mixtral are a compact general model and an eight-expert MoE. Both suit quick chat and tool use.

Quantisation: which version to take

• Q4 is the trade-off between memory and quality. Take it by default.
• Q8 gives almost the original quality at one and a half to two times the size of Q4. It makes sense if you have memory to spare.
• fp16 is full precision, three to four times the size of Q4. For chat and code it is usually not needed.
• Below 4 bits you save memory at the cost of a clear drop in quality. Not recommended for serious work.

Tags with suffixes such as q8_0 or fp16 usually name a specific quantisation. Check the tag's description in the library before you pull it.

What is inside a tag

Before you spend time downloading, look at what sits behind a tag. The ollama show command prints the architecture, the parameter count, the quantisation and the default context length.

ollama show qwen3:14b

To update a tag, run ollama pull again: Ollama downloads only the parts that changed. When you no longer need a tag, remove it and the disk space is freed.

How to choose a model in three steps

1. Work out the card's memory and the task: text, code, images or reasoning.
2. Take the largest tag from the table that fits with room for the context. For a baseline, use Q4.
3. Test the model on 10–20 of your own tasks. If the answers suit you, keep it; if not, move to a larger model or to another family.

This is quicker than comparing names. The size in the table tells you about memory, not quality: two models of the same size can differ noticeably in language and style.

How to pull a model

ollama pull qwen3:14b
ollama pull gemma3:12b
ollama list
ollama rm gemma3:12b   # frees disk space

On gpu.nz there are two ways. Run ollama pull in the server terminal, or enter the model name in the «Model» field of the template's launch form, using the same name as in Ollama, for example qwen3:8b. A Hugging Face model is pulled with a link of the form hf.co/organisation/repository:Q4_K_M.

How to call the API

Ollama serves a REST API, and its OpenAI-compatible routes live under /v1. From outside the server, the address is a protected link of the form https://<id>-11434.run.gpu.nz. Each request carries the access token from the server card in the Authorization header.

curl https://<id>-11434.run.gpu.nz/api/chat -H "Authorization: Bearer <token>" -d '{"model": "qwen3:14b", "stream": false, "think": false, "messages": [{"role": "user", "content": "Hello"}]}'

The think parameter set to false turns reasoning off for models that support it, such as Qwen3. The answer comes faster, but without the step-by-step breakdown. The /api/tags route lists the installed models.

Images as input

Models with image support, such as Gemma 3 from 4B up and Gemma 4, accept a picture together with the text. In the API the picture goes in as base64 in the images field of a message. A text-only model cannot answer about a picture, so check that the tag supports images.

curl https://<id>-11434.run.gpu.nz/api/chat -H "Authorization: Bearer <token>" -d '{"model": "gemma3:12b", "stream": false, "messages": [{"role": "user", "content": "What is shown in this picture?", "images": ["<picture in base64>"]}]}'

Default context

The context length is set by the OLLAMA_CONTEXT_LENGTH variable. In gpu.nz templates it is set to 8 thousand tokens in the Open WebUI + Ollama template, 16 thousand in most ready chats, and 32 thousand for GPT-OSS 120B. The bigger the context, the more memory the KV cache takes, so raise it deliberately and watch the memory with ollama ps.

Common mistakes

• The model does not fit: the processor column in ollama ps shows a share of CPU. Take a smaller quantisation or a card with more memory.
• The download stalls or breaks off: the files run to tens of gigabytes. Run ollama pull inside tmux or screen.
• The answers are odd or empty: check that you picked the chat tag, usually the instruct version, and not the base model.
• The disk is full: models take a lot of space. Remove the ones you do not use with ollama rm.

Where to start

If a model is new to you, start with 8 or 16 GB: at those sizes it is easier to see which tasks the model handles well. Then test the quality on your own examples, and only after that move to a bigger model. Minutes on a powerful card cost more, so choosing by memory saves money. You can pick a card in the GPU catalogue, and the API walkthrough is in the guide Ollama and vLLM on a rented GPU.