Open WebUI and Ollama on your own GPU server

What you can do

• a browser chat with open models: Qwen, Llama, Gemma, DeepSeek and others
• an OpenAI-format API for your apps, bots and scripts
• models from the Ollama library and GGUF files from Hugging Face
• switching between models inside one chat
• chat history kept on the server's disk until you delete the server
• no subscription: you pay only for the minutes the GPU runs

Video memory

8 GB+

What opens

Open WebUI chat and API

Which GPU you need

Most models in the Ollama library are 4-bit Q4 quantised: the weights take roughly 0.6 GB per billion parameters, plus memory for the context. An 8B model runs on 8–12 GB, for example an RTX 3060 12 GB, and a 14B model runs comfortably on 16 GB, such as an RTX 4080 16 GB. A 32B model needs 24 GB: an RTX 4090 24 GB or an RTX 3090. For a 70B model take a 48 GB card (RTX A6000, L40S) or an 80 GB one, such as an A100 80 GB.

How to start

1. In the catalog pick a card that fits your model and press «Rent».
2. In the launch sheet choose the Open WebUI + Ollama template. In the Model field enter a name such as qwen3:8b, or leave it empty and pull a model in the chat. Set the disk to at least 40 GB and press «Launch».
3. In a few minutes the server page shows an «Open the chat» button. The model usually downloads in 3–10 minutes, and a Telegram message tells you when the chat is ready.
4. For the API use the address https://<id>-11434.run.gpu.nz/v1, where <id> is the server code from the «Ollama API» link on its page. The key is the access token from the same page.
5. When you are done, stop the server and the GPU stops billing. If you no longer need the models, delete the server together with its disk.

Questions and answers

Other templates

• Qwen 3.8Alibaba's latest 27B model: code, writing, analysis, strong multilingual skills.
• GPT-OSS 20B UncensoredOpenAI's open model with refusals removed. Fast, fits in 16 GB of VRAM.
• vLLMFast LLM inference with an OpenAI-compatible API for your own apps.

Useful guides

• Run LLMs on a rented GPU: Llama, Qwen, DeepSeek, MistralRun Llama, Qwen, DeepSeek and Mistral on a rented GPU: Ollama with a chat UI or vLLM with an API. Which card you need, what an hour costs, and how to start.
• How much VRAM an LLM needs: a table and a rule of thumbHow to estimate memory for model weights at fp16, int8 and int4, how much 8B, 14B, 32B and 70B models need, and why context also uses memory.
• Ollama and vLLM on a rented GPU: chat and API in 10 minutesStep by step: a model in Ollama with chat and API, and vLLM with an OpenAI-compatible server on a rented GPU. Commands, checks and common mistakes.
• Local AI models without a powerful computerWhat a local AI model is, how much video memory text, image and video models need, and how to run the same open models on a rented GPU.
• Best Ollama models: what to run on 8, 16, 24, 48 and 80 GBWhich Ollama model to run on each amount of video memory: tags, file sizes, quantisation, pulling a model and the API. Qwen3, Gemma, Llama, DeepSeek R1.
• How to run DeepSeek locally or on your own serverThe full DeepSeek R1 and V3 need a multi-GPU server. Distilled R1 models from 1.5B to 70B run on one card: a memory table, Ollama and the API.