Run Qwen 3.8 27B on a VPS With Ollama

Can you run Qwen 3.8 27B on a VPS with no GPU?

To run Qwen 3.8 27B on a VPS you first need a model tag that exists, and as of 4 August 2026 the Ollama library has no qwen3.8 entry at all. The nearest released 27B tag is qwen3.6:27b: 27.8 billion parameters, Q4_K_M quantisation, Apache 2.0 licence. Every command and every number below uses that tag on Ollama v0.32.5, published 27 July 2026.

The short answer is yes on a 32 GB or larger VPS, and slowly. A 27B dense model at Q4 needs about 17 GB of RAM for weights alone, before a single token of context is stored. That rules out 8 GB and 16 GB plans completely. On a common two-channel DDR4 VPS the ceiling is roughly 3 tokens per second, which is slower than most people read.

Where does the 3.8 come from? Most likely the parameter count. The Ollama page for qwen3.6:27b reports 27.8B parameters, and 27.8 is easy to recall later as 3.8. There is also a qwen3.5:27b, the same Q4_K_M build from the previous release. Check the live list before you copy any command, at the Ollama qwen3.6 tag page. If a real qwen3.8 ships later, the arithmetic here still applies, because it depends on parameter count and bits per weight rather than on the version number.

Which Ollama tag to pull, and how to check

Pulling a tag that does not exist gives a clear error, so this is quick to settle on the box itself.

ollama pull qwen3.8:27b
# Error: pull model manifest: file does not exist

ollama pull qwen3.6:27b
ollama show qwen3.6:27b

ollama show prints the architecture, the parameter count, the context length and the quantisation for the tag you actually have. If the parameter line reads 27.8B and the quantisation line reads Q4_K_M, you have the build this guide is written against. The library also carries qwen3.6:27b-q8_0 and qwen3.6:27b-bf16 for the same weights at higher precision, plus a set of 35b-a3b tags that are MoE (mixture of experts) models and behave very differently on CPU. More on those below.

Parameter count times bytes per weight

[
  {
    "label": "Q4_K_M",
    "size_gb": 17,
    "bits_per_weight": 4.89,
    "notes": "published tag qwen3.6:27b"
  },
  {
    "label": "Q5_K_M",
    "size_gb": 19.8,
    "bits_per_weight": 5.7,
    "notes": "computed, no library tag exists"
  },
  {
    "label": "NVFP4",
    "size_gb": 20,
    "bits_per_weight": 5.76,
    "notes": "published tag 27b-nvfp4"
  },
  {
    "label": "Q8_0",
    "size_gb": 30,
    "bits_per_weight": 8.63,
    "notes": "published tag 27b-q8_0"
  },
  {
    "label": "BF16",
    "size_gb": 56,
    "bits_per_weight": 16.1,
    "notes": "published tag 27b-bf16"
  }
]

The formula is one line. Bytes of weights = parameters * bits per weight / 8. At a clean 4 bits, 27.8 billion parameters would be 13.9 GB. The shipped Q4_K_M tag is 17 GB, which works out at 4.89 bits per weight in practice.

That gap is not an error. K-quant formats do not store every tensor at the nominal width. The tensors that lose the most quality under compression are kept at 5 or 6 bits, and the token embedding and output layers are usually left at Q6_K or Q8_0. The name on the format is an average, and the average lands near 4.9. The same effect shows at the other end of the scale: 56 GB for BF16 is 16.1 bits per weight rather than a flat 16, because the file also carries metadata and a full-precision embedding table.

Q5_K_M has no published tag for this model, so the 19.8 GB row is computed at the usual 5.7 bits per weight for that format rather than measured. Q8_0 nearly doubles Q4 to 30 GB. On a CPU-only box that doubling costs twice the memory traffic per token, so it also roughly halves your tokens per second. Q4_K_M is the right default here for that reason alone.

What the KV cache costs as the context grows

The weights are a fixed cost. The KV cache (key and value cache, the attention state the model keeps for every token it has already seen) grows in a straight line with context length, and it is where most people actually run out of RAM.

[
  {
    "label": "4k tokens",
    "kv_f16_gb": 1,
    "kv_q8_gb": 0.5
  },
  {
    "label": "8k tokens",
    "kv_f16_gb": 2,
    "kv_q8_gb": 1
  },
  {
    "label": "16k tokens",
    "kv_f16_gb": 4,
    "kv_q8_gb": 2
  },
  {
    "label": "32k tokens",
    "kv_f16_gb": 8,
    "kv_q8_gb": 4
  },
  {
    "label": "64k tokens",
    "kv_f16_gb": 16,
    "kv_q8_gb": 8
  },
  {
    "label": "128k tokens",
    "kv_f16_gb": 32,
    "kv_q8_gb": 16
  }
]

Those figures assume the shape Qwen has used across its recent dense models in this size class: 64 layers, 8 key/value heads under GQA (grouped-query attention), and a head dimension of 128. That comes to 256 KiB per token at f16, so 8 GB at 32k tokens and 32 GB at 128k. Do not take my arithmetic over your own box. Load the model and read the SIZE column of ollama ps, which reports weights plus cache plus overhead as a single figure.

This is why the 256K context on the model card is a headline rather than a plan. Filling it at f16 would cost 64 GB of cache on top of the weights, on a machine that already spent 17 GB on weights. Ollama does not hand you the full window by default. It loads a much smaller one, and you raise it deliberately with OLLAMA_CONTEXT_LENGTH. Raise it in steps and check ollama ps after each change.

Two settings cut the cache in half or better. OLLAMA_KV_CACHE_TYPE=q8_0 stores the cache at 8 bits instead of 16, taking 32k tokens from 8 GB down to 4 GB. It needs flash attention, so set OLLAMA_FLASH_ATTENTION=1 as well, and confirm the drop in ollama ps instead of assuming it applied. OLLAMA_NUM_PARALLEL=1 matters just as much. Ollama can serve several requests at once, and each slot gets its own slice of context, so leaving parallelism at the default quietly multiplies the cache you budgeted for.

What fits in 8, 16, 32 and 64 GB of RAM

[
  {
    "label": "8 GB",
    "q4_max_ctx_ktok": 0,
    "q8_max_ctx_ktok": 0,
    "notes": "Weights alone exceed the box. Use a 4b or 8b model."
  },
  {
    "label": "16 GB",
    "q4_max_ctx_ktok": 0,
    "q8_max_ctx_ktok": 0,
    "notes": "17 GB of weights does not fit in 16 GB of RAM."
  },
  {
    "label": "32 GB",
    "q4_max_ctx_ktok": 32,
    "q8_max_ctx_ktok": 0,
    "notes": "Q4 fits with room to spare. Q8 weights do not fit."
  },
  {
    "label": "64 GB",
    "q4_max_ctx_ktok": 128,
    "q8_max_ctx_ktok": 64,
    "notes": "Both fit. Q8 leaves much less room for context."
  }
]

Read the two numbers as thousands of tokens of context that fit alongside the weights, at f16 cache, on a headless Linux VPS with about 1.5 GB left for the operating system and a small margin on top. A zero means the weights themselves do not fit, so nothing fits.

8 GB and 16 GB are not close calls. 17 GB of weights does not go into 16 GB of RAM, and no context setting changes that. Adding swap does not rescue it either. Ollama memory-maps the GGUF file, so once the resident pages exceed RAM the kernel starts evicting and re-reading them, and every token then pulls gigabytes off the disk. The box sits at high iowait and produces well under one token per second.

32 GB is the entry point. Weights take 17 GB and you have roughly 13 GB left, which covers about 32k tokens of f16 context with a margin. Q8_0 weights at 30 GB do not fit on this tier at all.

64 GB is comfortable. Q4 leaves room for around 128k tokens of context, and Q8_0 weights fit with about 64k tokens behind them. Before paying for 64 GB to get Q8, be clear about what you are buying: slightly better output at half the speed, on a machine that was already slow. Q4 with a longer context is the better trade for almost everyone.

How fast is CPU inference on a VPS?

Generating one token from a dense model means reading every weight from memory once. Not some of them. All of them. So the speed limit is not your core count, it is memory bandwidth divided by the size of the weights. At Q4 that is 17 GB of memory traffic per token.

[
  {
    "label": "DDR4-2666, 2 channel",
    "mem_bandwidth_gb_s": 42.6,
    "ceiling_tok_s": 2.5
  },
  {
    "label": "DDR4-3200, 2 channel",
    "mem_bandwidth_gb_s": 51.2,
    "ceiling_tok_s": 3
  },
  {
    "label": "DDR5-4800, 2 channel",
    "mem_bandwidth_gb_s": 76.8,
    "ceiling_tok_s": 4.5
  },
  {
    "label": "DDR4-3200, 8 channel",
    "mem_bandwidth_gb_s": 204.8,
    "ceiling_tok_s": 12
  },
  {
    "label": "DDR5-4800, 12 channel",
    "mem_bandwidth_gb_s": 460.8,
    "ceiling_tok_s": 27.1
  }
]

Those are ceilings, not measurements. Real output lands at roughly 50 to 70 percent of the figure shown, because memory latency and imperfect prefetching mean you never reach the theoretical peak. A two-channel DDR4-3200 VPS has a ceiling of 3 tokens per second, so expect about 2. A two-channel DDR5-4800 box has a ceiling of 4.5, so expect about 3.

The large server rows come with a warning. A twelve-channel EPYC platform has 460.8 GB/s and a ceiling of 27.1 tokens per second, but you do not rent a whole EPYC. Memory bandwidth is a host-wide resource shared by every tenant on that machine, so an 8 vCPU slice does not come with twelve channels of exclusive bandwidth. GPU-focused guides skip this entirely, and it is the reason two VPS plans with identical vCPU counts can differ by a factor of three on the same model.

More vCPUs stop helping early for the same reason. Once the cores request data faster than the memory controller can deliver it, extra threads add scheduling overhead and nothing else. Set OLLAMA_NUM_THREAD to your physical core count, measure, then try half that number. On many shared plans the lower setting is faster.

Prompt processing behaves differently. Prefill, the pass over your input before the first token appears, is compute bound rather than bandwidth bound, so it does scale with cores. The practical effect is a long pause before output starts on a large prompt, followed by the slow steady rate above. Time both halves separately with --verbose, which prints a prompt eval rate and an eval rate for every request.

If the dense 27B is simply too slow, look at the qwen3.6:35b-a3b tags before giving up on CPU. Those activate roughly 3 billion parameters per token instead of all 27.8 billion, so the memory traffic per token falls by close to an order of magnitude even though the file on disk is larger. You trade RAM footprint for speed. Runtime choice matters here too, and Ollama and llama.cpp expose different CPU tuning controls over the same underlying inference code.

When to rent a GPU hour instead

[
  {
    "label": "L40S, 48 GB",
    "mem_bandwidth_gb_s": 864,
    "ceiling_tok_s": 51
  },
  {
    "label": "RTX 4090, 24 GB",
    "mem_bandwidth_gb_s": 1008,
    "ceiling_tok_s": 59
  },
  {
    "label": "A100, 80 GB",
    "mem_bandwidth_gb_s": 2039,
    "ceiling_tok_s": 120
  },
  {
    "label": "H100 SXM, 80 GB",
    "mem_bandwidth_gb_s": 3350,
    "ceiling_tok_s": 197
  }
]

The same formula applied to published GPU memory bandwidth gives a different category of answer. A 24 GB consumer card has a ceiling of 59 tokens per second on these weights. A current data centre card reaches 197. That is not a gap you close by tuning thread counts. The card runs its memory at 1008 GB/s where your VPS runs at tens.

So draw the line by workload rather than by preference. CPU inference is the right answer when the work is asynchronous and nobody is waiting on it: overnight summarisation of a document pile, or a nightly classification job that runs while you sleep. Rent a GPU the moment a person is waiting for output, or the moment requests arrive faster than one every 30 seconds, because a CPU-only box has no batching headroom and the queue simply grows.

The cost comparison is less obvious than it looks. A 64 GB VPS bills every hour of the month whether the model is loaded or not, while a GPU instance bills only the hours you keep it running. If your real usage is two hours a day, the rented GPU can be both faster and cheaper. Work out your duty cycle first, then price it. Picking a VPS with a GPU covers what to check on the instance itself, and vLLM pulls ahead of Ollama once you serve concurrent requests on a GPU because it batches them properly.

There is a third option people forget. Keep the 27B on CPU for batch work and put a hosted API model in front of the interactive path. Nothing requires one model to serve both.

Install Ollama and measure your own box

The install script is the official one, and it sets up a systemd service running as a dedicated ollama user.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
free -g

ollama --version should print 0.32.5 or later. Check free -g before you pull anything. If the total column on the Mem line reads under 32, stop here and choose a smaller model, because pulling 17 GB you cannot run wastes an hour and a lot of disk.

Set the runtime options in a systemd override rather than in your shell. The model runs inside the service, so it never sees your interactive environment.

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_KEEP_ALIVE=60m"

sudo systemctl restart ollama
ollama pull qwen3.6:27b
ollama run qwen3.6:27b --verbose "Name two Linux distributions."

The --verbose output is the measurement you came for. eval rate is your tokens per second during generation. prompt eval rate is your prefill speed. load duration is how long it took to read the weights off disk, which is why OLLAMA_KEEP_ALIVE=60m is set: on CPU, reloading 17 GB from disk on every request costs more than the request itself.

While the model is loaded, check the footprint from a second terminal.

ollama ps

The SIZE column is the real memory footprint including the KV cache, and it should sit close to the weights plus the row for your context length in the KV chart. At 8192 tokens with an 8-bit cache, expect around a gigabyte on top of the weights, against 2 GB if the cache stayed at f16. The PROCESSOR column should read 100% CPU. If it reads anything else, something has claimed a GPU and the speed numbers in this guide do not describe your box.

Failure modes and the exact strings you will see

The model refuses to load. Ollama prints a line naming both figures, in the shape model requires more system memory (18.6 GiB) than is available (15.2 GiB). This is the good failure, because Ollama checked before allocating rather than letting the kernel sort it out. Lower the context length, drop to a smaller tag, or move to a larger plan.

The process disappears in the middle of an answer. The client shows nothing useful, and journalctl -u ollama -n 50 shows the service restarting. Run dmesg -T | tail, and a line reading Out of memory: Killed process ... (ollama) means the kernel OOM killer took it. That happens when the pre-load check passed but the cache grew past the estimate during a long conversation. Cut the context length.

The pull fails at once. Error: pull model manifest: file does not exist means the tag is not in the library. Typing qwen3.8:27b produces exactly this, and so does any typo in the version number. Confirm the tag on the library page before blaming your network.

Everything works but it is unbearably slow. Under one token per second on a box with enough RAM points at paging rather than at compute. Run vmstat 1 while generating. A non-zero si or so column means the kernel is swapping, and the fix is less context or fewer loaded models. Steady high wa with no swap activity means the memory-mapped weights are being re-read from disk, which means they do not really fit.

The first token takes 30 seconds and then output speeds up. That is prefill, and it is normal. A long system prompt is paid for on every request that misses the cache, so shorten the system prompt before tuning anything else.

What a CPU-only 27B is actually good for

Set expectations from the numbers rather than from hope. At two to four tokens per second, a 500 token answer takes between two and four minutes. That is unusable for chat and perfectly workable for a queue. Document summarisation, bulk tagging, field extraction from a backlog of files, and unattended code review all tolerate it, because nothing is waiting on the reply.

The privacy argument is the real one. The model runs on hardware you rent and control, no request leaves the box, and there is no per-token bill. That is worth a lot for regulated data even at three tokens per second. Weigh it honestly against the alternative: self-hosting a frontier-scale model needs an order of magnitude more hardware, and a 27B on CPU is the cheapest point on that curve where the output is still worth reading.

If this is your first Ollama install, the full walkthrough for running Ollama on a VPS covers the service setup, the HTTP API and the firewall rules this guide assumes you already have. Do not expose port 11434 to the internet. Ollama ships with no authentication of its own, so anything that reaches the port can use your model and read your prompts.

FAQ

Is there a Qwen 3.8 27B model on Ollama?

No. As of 4 August 2026 the Ollama library has no qwen3.8 namespace. The 27B tags that exist are qwen3.5:27b and qwen3.6:27b, both Q4_K_M builds of a 27.8 billion parameter dense model. The 3.8 in the search term is almost certainly the 27.8B parameter count remembered as a version number. Check https://ollama.com/library/qwen3.6/tags for the current list, and pull qwen3.6:27b if you want the newest released 27B. A tag that does not exist fails with Error: pull model manifest: file does not exist.

How much RAM do I need to run a Qwen 27B model on a VPS?

32 GB is the practical minimum for Q4_K_M. The weights are 17 GB, the operating system needs around 1.5 GB, and the KV cache adds roughly 1 GB for every 4000 tokens of context at f16. A 16 GB plan cannot hold the weights at all, and swap does not help because the file is memory-mapped and the kernel just re-reads it from disk on every token. 64 GB gives you room for long context or for Q8_0 weights at 30 GB.

How many tokens per second will a 27B model give me on CPU?

Divide your memory bandwidth by the size of the weights, then take 50 to 70 percent of that. A two-channel DDR4-3200 VPS has a ceiling near 3 tokens per second and delivers about 2. A two-channel DDR5-4800 box has a ceiling near 4.5 and delivers about 3. Higher-channel server platforms look much better on paper, but memory bandwidth is shared across every tenant on the host, so measure your own with ollama run qwen3.6:27b --verbose and read the eval rate line.

Should I use Q4 or Q8 on a CPU-only VPS?

Q4_K_M, in almost every case. Q8_0 is 30 GB against 17 GB, so it needs a 64 GB plan and it moves nearly twice as much memory per token, which cuts your tokens per second roughly in half. The quality difference between Q4_K_M and Q8_0 on a 27B model is small for most tasks. Spend the RAM on a longer context instead, since that changes what the model can do rather than how it phrases things.

When is renting a GPU cheaper than a big RAM VPS?

When your duty cycle is low or a person is waiting. A GPU with 24 GB of memory reaches roughly 59 tokens per second on these weights, against 2 or 3 on a typical VPS, and it bills only for the hours it runs. A 64 GB VPS bills all month whether the model is loaded or not. Work out how many hours a day you really generate tokens. Under two or three hours, hourly GPU rental usually wins on both speed and cost. Continuous low-priority batch work is where the always-on VPS wins.