Run Gemma 4 on a VPS: sizing and speed

Will Gemma 4 fit on your VPS

Gemma 4 fits on a VPS when you match the tag to the RAM you have, and the smaller tags are the realistic ones. On a CPU-only server, gemma4:e4b-it-qat at 6.1 GB is comfortable on 8 GB of RAM, and gemma4:12b-it-qat at 7.2 GB is comfortable on 16 GB. The 26b and 31b tags need a 32 GB box before they load at all, and only one of those two answers fast enough to be worth using without a graphics card.

Gemma 4 is a family of open models from Google DeepMind, published on ollama.com under the Apache 2.0 licence. Every size below was read from the model's tag pages on 31 August 2026, running Ollama v0.33.2, released 27 August 2026. Tag names are the thing that changes most between model releases, so check ollama.com/library/gemma4/tags yourself before copying a pull command out of any guide, this one included. ollama -v prints the version you have.

If Ollama is not on the box yet, the install and first model walkthrough covers the service setup this post assumes.

What tags does Gemma 4 ship

There are five local sizes and one hosted one. E2B and E4B are the edge sizes, where the "E" stands for effective parameters, and both carry a 128K token context. The 12b, 26b and 31b sizes carry a 256K token context. The 26b is a mixture of experts (MoE) model: 25.2B parameters in total, of which 3.8B are active for any single token. The 31b is dense at 30.7B parameters, so all of them are active on every token. That one difference decides your speed, and it comes back later in this post.

Each size has a default Q4_K_M tag plus alternatives named -it-qat, -it-q8_0, -it-bf16, -nvfp4, -mxfp8 and -mlx. QAT means quantisation-aware training: the model is trained with the low-bit format already in the loop instead of being compressed after training. The QAT tags are smaller on disk than the plain Q4_K_M tags at the same parameter size, so they are the first thing to try when RAM is your limit. The -mlx tags are built for Apple Silicon and do nothing for you on a Linux VPS. For what the quantisation letters cost in answer quality, read the Q4 against Q8 against FP16 comparison.

One trap before you type anything: gemma4:latest is E4B, the 4.5B effective edge model. ollama run gemma4 with no tag does not give you the big one.

[
  {
    "label": "e2b-it-qat",
    "download_gb": 4.3,
    "active_params_b": 2.3
  },
  {
    "label": "e4b-it-qat",
    "download_gb": 6.1,
    "active_params_b": 4.5
  },
  {
    "label": "12b-it-qat",
    "download_gb": 7.2,
    "active_params_b": 11.9
  },
  {
    "label": "12b q4_K_M",
    "download_gb": 7.6,
    "active_params_b": 11.9
  },
  {
    "label": "26b q4_K_M MoE",
    "download_gb": 19.0,
    "active_params_b": 3.8
  },
  {
    "label": "31b q4_K_M",
    "download_gb": 20.0,
    "active_params_b": 30.7
  },
  {
    "label": "31b-it-q8_0",
    "download_gb": 34.0,
    "active_params_b": 30.7
  },
  {
    "label": "31b-it-bf16",
    "download_gb": 63.0,
    "active_params_b": 30.7
  }
]

The smallest tag worth running is gemma4:e2b-it-qat at 4.3 GB. The largest is gemma4:31b-it-bf16 at 63.0 GB, which is more disk than many VPS plans ship in total. Then look at the two middle rows. The 26b at 19.0 GB and the 31b at 20.0 GB sit within a gigabyte of each other on disk, while their active parameter counts are 3.8B and 30.7B. They cost nearly the same to store. They are nothing alike to run.

The sizing arithmetic you can redo yourself

The number on a tag page is the download. Memory while the model answers is larger, and it has two moving parts: the weights, and the KV cache (key/value cache, the per-token attention state kept for the conversation so far). Runtime overhead adds a few hundred megabytes on top of both.

You could compute the cache from layer counts and head dimensions, but the Gemma 4 details block lists a sliding window of 1024 tokens, which means not every layer holds the whole context, so the textbook formula overestimates. Measure it instead. Two loads and one subtraction give you the real figure for the tag you picked.

sudo systemctl edit ollama.service

Add this block, save it, then restart the service and load the model once.

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=4096"

sudo systemctl restart ollama
ollama run gemma4:12b-it-qat "hi"
ollama ps

ollama ps lists the loaded model with a SIZE column and a PROCESSOR column. Write down SIZE. Now change the drop-in to OLLAMA_CONTEXT_LENGTH=32768, restart, run the same prompt, and write down SIZE again. The difference divided by 28672, which is 32768 minus 4096, is the cache cost per token for that tag. Multiply that by the context you actually want, add the weights, and you have your RAM requirement.

Two more terms belong in that multiplication. Concurrency is one: Ollama serves parallel requests from separate cache slots, so OLLAMA_NUM_PARALLEL multiplies the cache, which is the subject of how the parallel and queue settings interact. Cache precision is the other: OLLAMA_KV_CACHE_TYPE=q8_0 roughly halves the cache against the default of f16. Verify it by comparing SIZE in ollama ps before and after. If SIZE does not move, the setting did not apply and you are still paying the full price.

Context length is the setting people forget. Ollama does not hand you the model's full 256K window by default, because the documented defaults scale with available video RAM (VRAM) and start at 4k. A CPU-only VPS lands at the bottom of that scale, so set the number explicitly rather than assuming, and read what num_ctx really costs in memory before you raise it.

Install, pull, and check what you got

Run these on your own server. The pull is a large download, so check the disk first rather than after.

curl -fsSL https://ollama.com/install.sh | sh
ollama -v
df -h /usr/share/ollama
ollama pull gemma4:12b-it-qat
ollama ls

ollama ls should list the tag with its size. On Linux the blobs land in /usr/share/ollama/.ollama/models, which is on the root volume, and OLLAMA_MODELS moves them elsewhere. A 20 GB pull onto a 40 GB VPS disk leaves very little room for logs, backups or anything else you host. Where Ollama stores models and what pull actually downloads covers the layout.

Then time it.

ollama run gemma4:12b-it-qat --verbose "Explain what a reverse proxy does, in two sentences."

--verbose prints a timing block after the answer. The line that matters is eval rate, the decode speed in tokens per second. prompt eval rate is prefill, meaning how fast your input was read. Check ollama ps straight after: a PROCESSOR column reading 100% CPU means no graphics card was used, which is expected on a CPU-only plan and a fault to chase on a GPU plan. Measuring tokens per second properly explains how to make that figure repeatable.

How slow is Gemma 4 on a CPU-only VPS

Slow, and you can predict how slow before you pull anything. Producing one token requires reading every active weight out of RAM once, so CPU decode is limited by memory bandwidth rather than by clock speed. The ceiling is bandwidth divided by bytes read per token. Get a rough bandwidth figure for your box first.

sudo apt update && sudo apt install -y sysbench
sysbench memory --memory-block-size=1M --memory-total-size=10G run

The chart below assumes 12 GB/s, a plausible read figure for a few shared virtual cores. Put your own number in and redo the division. The MoE row uses 19 GB scaled by 3.8 active parameters out of 25.2 total, which is about 2.9 GB read per token.

[
  {
    "label": "e4b-it-qat",
    "gb_read_per_token": 6.1,
    "tok_s_ceiling": 2.0
  },
  {
    "label": "12b-it-qat",
    "gb_read_per_token": 7.2,
    "tok_s_ceiling": 1.7
  },
  {
    "label": "26b, 3.8B active",
    "gb_read_per_token": 2.9,
    "tok_s_ceiling": 4.1
  },
  {
    "label": "31b, 30.7B active",
    "gb_read_per_token": 20.0,
    "tok_s_ceiling": 0.6
  }
]

The 31b dense tag reads 20.0 GB per token and tops out near 0.6 tokens per second. The 26b MoE tag touches only its active experts, about 2.9 GB per token, so it ceilings near 4.1 tokens per second from a download of almost the same size. That is why the MoE architecture matters on a CPU box: you pay for the whole model in RAM, and you pay for the active slice in time.

Two caveats, stated plainly. This is a ceiling and not a measurement, so real output lands below it, because attention work and sampling are not free. And the E rows are pessimistic, because those downloads bundle vision and audio encoders that are not read on every decode token.

Now add thinking. Gemma 4 has configurable thinking modes, switched on by putting a <|think|> token at the start of the system prompt, and a thinking model spends tokens reasoning before it writes anything you see. At 0.6 tokens per second, 800 thinking tokens is more than twenty minutes of silence before the first word of the reply. Thinking is a feature you can afford on a GPU and usually cannot afford on a CPU-only VPS.

Which VPS tier for which tag

• 8 GB RAM, CPU only: gemma4:e2b-it-qat or gemma4:e4b-it-qat at a 4k to 8k context. Good for short replies and text extraction.
• 16 GB RAM, CPU only: gemma4:12b-it-qat at 8k to 16k. Around 2 tokens per second by the arithmetic above, so it suits background jobs and frustrates anyone chatting with it.
• 32 GB RAM, CPU only: gemma4:26b. It loads with room left for real context and it is the only large tag with a tolerable CPU speed. gemma4:31b also loads on this tier, and then crawls.
• GPU with 24 GB VRAM: gemma4:12b sits in VRAM with a large context. The Q4 26b and 31b tags fit at around 20.0 GB, which leaves only a few gigabytes for the cache, so keep the context small.
• GPU with 48 GB VRAM or more: gemma4:31b-it-q8_0 at 34.0 GB, or the Q4 tag with a context worth having.

On every tier, set the idle timeout deliberately. Ollama unloads a model after five minutes by default, and reloading 19 GB from disk adds a long pause to the next request, so keeping the model resident between requests is worth doing on a box with the RAM to spare. If you are choosing between model families rather than tiers, the same sizing walk for Qwen 3.8 27B and the wider list of models that fit a VPS are the next comparisons to read.

When Gemma 4 is the wrong tool for your box

gemma4:31b on a CPU-only VPS is the clearest case. It loads on 32 GB and then writes slower than you read, so the machine is busy for minutes per reply and can serve one person. The 26b MoE gives you most of the capability at several times the speed for the same disk. Choose the 31b when you have VRAM, not before.

The -it-bf16 tags are not VPS tags at all. gemma4:31b-it-bf16 is 63.0 GB of weights, so it wants an 80 GB accelerator. Renting one by the hour to serve a model that runs at Q4 on hardware you already have is a spending decision rather than a technical one.

Image work is the third case. Gemma 4 accepts images with a visual token budget between 70 and 1120 tokens each, and the image goes before the text in the prompt. Those tokens are prefill, and prefill on a CPU is slow, so a document page that takes a second on a GPU can take a minute on a CPU-only VPS.

Serving a group of people is the fourth. One instance on one CPU box answers one request at a time in any useful sense, because parallel slots split the same memory bandwidth between them. What concurrency really costs on a self-hosted model has that arithmetic. If the box will serve anyone other than you, put the endpoint behind authentication first, since Ollama ships with none: locking down the Ollama API covers it.

What you see when it does not fit

Ask for more than the box has and the model still tries to load. Linux starts swapping, tokens per second drops to near zero, and eventually the out of memory (OOM) killer stops the runner. The client prints an error naming the runner process terminating with signal: killed, journalctl -u ollama -n 50 shows the same event from the server side, and dmesg -T | grep -i "out of memory" confirms the kernel did it.

The fix is a smaller number somewhere. Drop OLLAMA_CONTEXT_LENGTH first, because context is the cheapest thing to give up and often the reason you overran. Next switch to the -it-qat tag at the same parameter size, which costs a gigabyte or two of RAM and very little quality. Move down a size only after both of those. Adding swap is not a fix: swap makes the weights readable, not fast, and a model paging from disk on every token is not a service anyone can use.

FAQ

How much RAM does Gemma 4 need on a VPS?

Take the tag's download size and add the KV cache. gemma4:e4b-it-qat at 6.1 GB runs on 8 GB of RAM, gemma4:12b-it-qat at 7.2 GB runs on 16 GB, and the Q4 26b and 31b tags at 19.0 GB and 20.0 GB need 32 GB. Those sizes were read from ollama.com on 31 August 2026, so check the tag page for current ones. Measure your own cache cost by loading the model once with OLLAMA_CONTEXT_LENGTH=4096 and once with 32768, then subtracting the two SIZE values that ollama ps reports.

Which Gemma 4 tag is fastest on a CPU-only VPS?

Among the large tags, gemma4:26b. It is a mixture of experts model with 3.8B active parameters out of 25.2B total, and CPU decode speed follows active parameters, because the processor reads those weights out of RAM for every token it writes. gemma4:31b is dense with 30.7B active, so it reads roughly seven times as much per token from a download only one gigabyte larger. On smaller boxes the E tags are faster still, and gemma4:e4b-it-qat is the usual pick at 8 GB.

Why does Ollama not give me the full 256K context?

Because the default context is much smaller and scales with available VRAM, starting around 4k, and a 256K window would need a KV cache far larger than the weights themselves on any VPS. Set OLLAMA_CONTEXT_LENGTH in a systemd drop-in for ollama.service, restart the service, and read the SIZE column in ollama ps to see what the increase cost you. If SIZE barely changes when you raise the number, the drop-in did not take effect.

Do I need a GPU to run Gemma 4?

No for the E tags and the 12b, and yes in practice for the 31b. On CPU the memory bandwidth ceiling is roughly 1.7 tokens per second for gemma4:12b-it-qat and about 0.6 for the 31b dense tag, which is workable for background jobs and unpleasant for chat. A card with 24 GB of VRAM runs the 12b comfortably, and the Q4 26b or 31b with a small context.