How to run Qwen 27B model for VPS with Ollama

You fit run Qwen 3.8 27B for VPS wey no get GPU?

To run Qwen 3.8 27B for VPS, you must first get model tag wey dey exist, and as of 4 August 2026, the Ollama library no get any qwen3.8 entry at all. The closest released 27B tag na qwen3.6:27b: 27.8 billion parameters, Q4_K_M quantisation, Apache 2.0 licence. Every command and every number wey dey below use that tag for Ollama v0.32.5, wey dem publish 27 July 2026.

The short answer na yes, if you get 32 GB or bigger VPS, but e go slow. One 27B dense model for Q4 need about 17 GB of RAM for weights alone, before e even store one single token of context. This one mean say 8 GB and 16 GB plans no fit work at all. For one common two-channel DDR4 VPS, the limit na roughly 3 tokens per second, wey slow pass how most people dey read.

Where that 3.8 come from? E fit be the parameter count. The Ollama page for qwen3.6:27b show 27.8B parameters, and 27.8 easy to remember later as 3.8. E still get one qwen3.5:27b, the same Q4_K_M build from the previous release. Check the live list before you copy any command, for the Ollama qwen3.6 tag page. If one real qwen3.8 ship later, the calculation wey dey here still valid, because e depend on parameter count and bits per weight, no be the version number.

Which Ollama tag to pull, and how to check

If you pull one tag wey no exist, you go see clear error, so e easy to know wetin dey ground for your box. Even if the tag exist, e fit still no run for your local machine, and na this one dey confuse people with GLM 5.2, wey dey for library but na only from Ollama cloud dem dey serve am.

ollama pull qwen3.8:27b
# Error: pull model manifest: file does not exist

ollama pull qwen3.6:27b
ollama show qwen3.6:27b

ollama show go print the architecture, the parameter count, the context length, and the quantisation for the tag wey you get. If the parameter line show 27.8B and the quantisation line show Q4_K_M, e mean say you get the build wey this guide dey use. The library still get qwen3.6:27b-q8_0 and qwen3.6:27b-bf16 for the same weights wey get higher precision, plus one set of 35b-a3b tags wey be MoE (mixture of experts) models and dem dey behave differently for CPU. We go talk more about dem below.

Calculation of parameter count times bytes per weight

[
  {
    "label": "Q4_K_M",
    "size_gb": 17,
    "bits_per_weight": 4.89,
    "notes": "published tag qwen3.6:27b"
  },
  {
    "label": "Q5_K_M",
    "size_gb": 19.8,
    "bits_per_weight": 5.7,
    "notes": "computed, no library tag exists"
  },
  {
    "label": "NVFP4",
    "size_gb": 20,
    "bits_per_weight": 5.76,
    "notes": "published tag 27b-nvfp4"
  },
  {
    "label": "Q8_0",
    "size_gb": 30,
    "bits_per_weight": 8.63,
    "notes": "published tag 27b-q8_0"
  },
  {
    "label": "BF16",
    "size_gb": 56,
    "bits_per_weight": 16.1,
    "notes": "published tag 27b-bf16"
  }
]

Di formula na one line. Bytes of weights = parameters * bits per weight / 8. For clean 4 bits, 27.8 billion parameters go be 13.9 GB. Di Q4_K_M tag wey dem ship na 17 GB, wey come out to 4.89 bits per weight for real life.

Dat gap no be error. K-quant formats no dey store every tensor for di nominal width. Dem dey keep di tensors wey fit lose quality pass under compression for 5 or 6 bits, and dem usually leave di token embedding and output layers for Q6_K or Q8_0. Di name wey dey di format na average, and di average dey near 4.9. Di same effect dey show for di oda side of di scale: 56 GB for BF16 na 16.1 bits per weight instead of just 16, sake of say di file still carry metadata and full-precision embedding table.

Q5_K_M no get published tag for dis model, so dem calculate di 19.8 GB row for di usual 5.7 bits per weight for dat format instead of say dem measure am. Q8_0 almost double Q4 reach 30 GB. For CPU-only box, dat doubling dey cost twice di memory traffic per token, so e almost cut your tokens per second for half. Q4_K_M na di correct default for here sake of dat reason alone. If you want di quality side of dat decision instead of di memory side, one closer comparison of Q4, Q8 and fp16 show where di output actually start to spoil.

How much KV cache dey cost as context dey grow

The weights na fixed cost. The KV cache (key and value cache, the attention state wey the model dey keep for every token wey e don see before) dey grow straight line as context length dey increase, and na here most pipo dey run out of RAM.

[
  {
    "label": "4k tokens",
    "kv_f16_gb": 1,
    "kv_q8_gb": 0.5
  },
  {
    "label": "8k tokens",
    "kv_f16_gb": 2,
    "kv_q8_gb": 1
  },
  {
    "label": "16k tokens",
    "kv_f16_gb": 4,
    "kv_q8_gb": 2
  },
  {
    "label": "32k tokens",
    "kv_f16_gb": 8,
    "kv_q8_gb": 4
  },
  {
    "label": "64k tokens",
    "kv_f16_gb": 16,
    "kv_q8_gb": 8
  },
  {
    "label": "128k tokens",
    "kv_f16_gb": 32,
    "kv_q8_gb": 16
  }
]

Those figures assume the shape wey Qwen don use across im recent dense models for this size class: 64 layers, 8 key/value heads under GQA (grouped-query attention), and head dimension of 128. That one come to 256 KiB per token for f16, so 8 GB for 32k tokens and 32 GB for 128k. No just follow my calculation, check your own machine. Load the model and read the SIZE column of ollama ps, wey dey report weights plus cache plus overhead as one figure.

This na why the 256K context wey dey the model card na just headline, no be plan. To fill am for f16 go cost 64 GB of cache join the weights, for machine wey don already spend 17 GB for weights. Ollama no dey give you the full window by default. E dey load small one, and you go increase am intentionally with OLLAMA_CONTEXT_LENGTH. That server-wide variable no be the only way, and setting num_ctx for individual request go allow you keep cheap default for everything else while one long job get the bigger window. Increase am small-small and check ollama ps after every change.

Two settings fit cut the cache for half or even pass. OLLAMA_KV_CACHE_TYPE=q8_0 dey store the cache for 8 bits instead of 16, e dey bring 32k tokens from 8 GB down to 4 GB. E need flash attention, so set OLLAMA_FLASH_ATTENTION=1 join am, and confirm the drop for ollama ps instead of assuming say e don work. OLLAMA_NUM_PARALLEL=1 dey important well-well. Ollama fit serve many requests at once, and every slot get im own slice of context, so if you leave parallelism for default, e go multiply the cache wey you budget. If more than one person go use this box, that multiplication na where the wahala dey start, and the number of concurrent users wey self-hosted model fit serve na cache slots and queue depth dey decide am, long before core count come matter.

Wetin fit enta 8, 16, 32 and 64 GB of RAM

[
  {
    "label": "8 GB",
    "q4_max_ctx_ktok": 0,
    "q8_max_ctx_ktok": 0,
    "notes": "Weights alone exceed the box. Use a 4b or 8b model."
  },
  {
    "label": "16 GB",
    "q4_max_ctx_ktok": 0,
    "q8_max_ctx_ktok": 0,
    "notes": "17 GB of weights does not fit in 16 GB of RAM."
  },
  {
    "label": "32 GB",
    "q4_max_ctx_ktok": 32,
    "q8_max_ctx_ktok": 0,
    "notes": "Q4 fits with room to spare. Q8 weights do not fit."
  },
  {
    "label": "64 GB",
    "q4_max_ctx_ktok": 128,
    "q8_max_ctx_ktok": 64,
    "notes": "Both fit. Q8 leaves much less room for context."
  }
]

Make you read di two numbers as thousands of tokens of context wey fit stay join di weights, for f16 cache, on top one headless Linux VPS wey get about 1.5 GB free for operating system plus small margin. Zero mean say di weights no fit enta at all, so nothing fit work.

8 GB and 16 GB no be small matter. 17 GB of weights no fit enta 16 GB of RAM, and no context setting fit change dat one. Even if you add swap, e no go save you. Ollama dey memory-map di GGUF file, so once di resident pages pass di RAM, di kernel go start to dey evict and re-read dem, and every token go come dey pull gigabytes from disk. Di machine go just dey high iowait and e go dey produce well under one token per second.

32 GB na di starting point. Weights go take 17 GB and you go get roughly 13 GB left, wey fit cover about 32k tokens of f16 context with small margin. Q8_0 weights wey be 30 GB no fit enta this tier at all.

64 GB dey comfortable. Q4 go leave space for about 128k tokens of context, and Q8_0 weights fit enta with about 64k tokens behind dem. Before you pay for 64 GB just to get Q8, make you clear wetin you dey buy: slightly better output for half di speed, on top machine wey already slow. Q4 with longer context na better trade for almost everybody.

How fast CPU inference dey for VPS?

To generate one token from one dense model mean say you must read every weight from memory one time. No be some of dem. Na all of dem. So, the speed limit no be how many core you get, na memory bandwidth wey you divide by the size of the weights. For Q4, that one na 17 GB of memory traffic per token.

[
  {
    "label": "DDR4-2666, 2 channel",
    "mem_bandwidth_gb_s": 42.6,
    "ceiling_tok_s": 2.5
  },
  {
    "label": "DDR4-3200, 2 channel",
    "mem_bandwidth_gb_s": 51.2,
    "ceiling_tok_s": 3
  },
  {
    "label": "DDR5-4800, 2 channel",
    "mem_bandwidth_gb_s": 76.8,
    "ceiling_tok_s": 4.5
  },
  {
    "label": "DDR4-3200, 8 channel",
    "mem_bandwidth_gb_s": 204.8,
    "ceiling_tok_s": 12
  },
  {
    "label": "DDR5-4800, 12 channel",
    "mem_bandwidth_gb_s": 460.8,
    "ceiling_tok_s": 27.1
  }
]

Those numbers na maximum, no be wetin you go see for ground. Real output dey land for roughly 50 to 70 percent of the figure wey dem show, because memory latency and imperfect prefetching mean say you no go ever reach the theoretical peak. One two-channel DDR4-3200 VPS get ceiling of 3 tokens per second, so expect about 2. One two-channel DDR5-4800 box get ceiling of 4.5, so expect about 3.

The big server rows come with warning. One twelve-channel EPYC platform get 460.8 GB/s and ceiling of 27.1 tokens per second, but you no dey rent full EPYC. Memory bandwidth na resource wey all tenant for that machine dey share, so one 8 vCPU slice no come with twelve channels of exclusive bandwidth. Guides wey focus on GPU dey skip this matter entirely, and na the reason why two VPS plans wey get same vCPU count fit differ by factor of three for the same model.

More vCPUs no dey help again once you reach certain point for the same reason. Once the cores dey request data faster pass wetin the memory controller fit deliver, extra threads just dey add scheduling overhead and nothing else. Set OLLAMA_NUM_THREAD to your physical core count, measure, then try half that number. For many shared plans, the lower setting dey faster.

Prompt processing dey behave differently. Prefill, the pass wey you do over your input before the first token show, na compute bound instead of bandwidth bound, so e dey scale with cores. The practical effect na long pause before output start for large prompt, followed by the slow steady rate wey we mention up there. Time both halves separately with --verbose, wey dey print prompt eval rate and eval rate for every request.

If the dense 27B just too slow, look at the qwen3.6:35b-a3b tags before you give up on CPU. Those ones dey activate roughly 3 billion parameters per token instead of all 27.8 billion, so the memory traffic per token go drop by close to one order of magnitude even though the file on disk big pass. You dey trade RAM footprint for speed. Runtime choice matter here too, and Ollama and llama.cpp expose different CPU tuning controls over the same underlying inference code.

When to rent a GPU hour instead

[
  {
    "label": "L40S, 48 GB",
    "mem_bandwidth_gb_s": 864,
    "ceiling_tok_s": 51
  },
  {
    "label": "RTX 4090, 24 GB",
    "mem_bandwidth_gb_s": 1008,
    "ceiling_tok_s": 59
  },
  {
    "label": "A100, 80 GB",
    "mem_bandwidth_gb_s": 2039,
    "ceiling_tok_s": 120
  },
  {
    "label": "H100 SXM, 80 GB",
    "mem_bandwidth_gb_s": 3350,
    "ceiling_tok_s": 197
  }
]

If you look at the memory bandwidth wey dem publish for GPU, the answer go change. One 24 GB consumer card get limit of 59 tokens per second for these weights. One current data centre card fit reach 197. You no fit close this gap just by tuning thread counts. The card dey run im memory at 1008 GB/s, but your VPS dey run for tens of GB/s.

So, make you choose based on the work wey you get, no be based on wetin you like. CPU inference na the correct answer if the work no need immediate response and nobody dey wait for am: like when you wan summarize plenty document for night, or run classification job while you dey sleep. Rent one GPU the moment person dey wait for output, or if requests dey come faster than one every 30 seconds, because CPU-only box no get space for batching and the queue go just dey grow.

The cost comparison no dey as simple as e look. One 64 GB VPS go charge you every hour for the month whether the model dey load or not, but GPU instance go charge you only for the hours wey you keep am running. If you only use am for two hours per day, the rented GPU fit be faster and cheaper. Calculate your duty cycle first, then check the price. Picking a VPS with a GPU explain wetin you suppose check for the instance, and vLLM pulls ahead of Ollama once you serve concurrent requests on a GPU because e dey batch requests well.

One third option dey wey people dey forget. Keep the 27B on CPU for batch work and use hosted API model for the interactive path. Nothing talk say na only one model must do both work.

Install Ollama and measure your own box

The install script na the official one, and e dey set up one systemd service wey dey run as one dedicated ollama user.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
free -g

ollama --version suppose print 0.32.5 or later. Check free -g before you pull anything. If the total column for the Mem line show number wey less than 32, stop here and choose one smaller model, because to pull 17 GB wey you no fit run na waste of one hour and plenty disk space.

Set the runtime options inside one systemd override instead of your shell. The model dey run inside the service, so e no dey ever see your interactive environment.

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_KEEP_ALIVE=60m"

sudo systemctl restart ollama
ollama pull qwen3.6:27b
ollama run qwen3.6:27b --verbose "Name two Linux distributions."

The --verbose output na the measurement wey you come find. eval rate na your tokens per second during generation. prompt eval rate na your prefill speed. load duration na how long e take to read the weights from disk, and na why OLLAMA_KEEP_ALIVE=60m dey set: for CPU, to reload 17 GB from disk for every request dey cost pass the request itself. The default idle timeout na five minutes, e short reach say one batch queue wey get gap between items go pay that load cost over and over, and the options for keeping a model resident cover both the per-request keep_alive field and how to make the setting survive one reboot.

While the model dey loaded, check the footprint from one second terminal.

ollama ps

The SIZE column na the real memory footprint wey include the KV cache, and e suppose dey close to the weights plus the row for your context length inside the KV chart. At 8192 tokens with one 8-bit cache, expect around one gigabyte on top of the weights, against 2 GB if the cache stay for f16. The PROCESSOR column suppose read 100% CPU. If e read anything else, e mean say something don claim one GPU and the speed numbers for this guide no describe your box.

Ways wey tin fit fail and the exact message wey you go see

The model no gree load. Ollama go print one line wey mention the two figures, e go look like model requires more system memory (18.6 GiB) than is available (15.2 GiB). This one na better failure, because Ollama check am before e allocate memory, instead of make you leave am for kernel to handle. Reduce the context length, use smaller tag, or upgrade to better plan.

The process just disappear while e dey answer. The client no go show anything wey make sense, and journalctl -u ollama -n 50 go show say the service dey restart. Run dmesg -T | tail, and if you see one line wey read Out of memory: Killed process ... (ollama), e mean say the kernel OOM killer don kill the process. This one dey happen when the pre-load check pass, but the cache come big pass wetin dem estimate during long conversation. Reduce the context length.

The pull fail sharp-sharp. Error: pull model manifest: file does not exist mean say that tag no dey the library. If you type qwen3.8:27b, you go get this same error, and any small mistake for the version number go cause am too. Confirm the tag for the library page before you blame your network.

Everything dey work but e too slow. If you dey get less than one token per second for machine wey get enough RAM, e mean say paging dey happen, no be say the compute dey slow. Run vmstat 1 while e dey generate. If the si or so column no be zero, e mean say the kernel dey swap, and the fix na to reduce context or reduce the number of models wey you load. If wa dey high steady and swap no dey happen, e mean say the memory-mapped weights dey read from disk again, which mean say dem no fit fit inside memory.

The first token take 30 seconds before e start to run fast. That one na prefill, and e normal. You dey pay for long system prompt for every request wey no hit the cache, so shorten your system prompt before you try change anything else.

Wetin CPU-only 27B model actually dey good for

Make your expectation base on numbers, no be just hope. With two to four tokens per second, 500 tokens go take between two and four minutes to finish. This one no fit work for chat, but e dey perfect for queue. Model wey dey think before e reply go make the math worse, because the hidden reasoning tokens dey generate for the same slow speed as the answer. So, make the reasoning effort level match the job na one of the few ways to make reply fast without changing the model. Document summarisation, bulk tagging, field extraction from files wey don pile up, and unattended code review all fit tolerate this speed, because nothing dey wait for the reply. Coding assistance dey for the middle, so point your coding agent to a model wey you host dey pay for background jobs like commit messages and test scaffolding, but e no go work for inline suggestions wey you go sit down dey wait for.

The privacy argument na the main point. The model dey run for hardware wey you rent and control, no request dey leave the box, and you no get per-token bill. This one get value for regulated data even if the speed na three tokens per second. Weigh am well against the alternative: self-hosting a frontier-scale model need ten times more hardware, and 27B on CPU na the cheapest point for that curve where the output still dey worth to read.

If this na your first time to install Ollama, the full walkthrough for running Ollama on a VPS don cover how to setup the service, the HTTP API, and the firewall rules wey this guide assume say you don already get. No expose port 11434 to the internet. Ollama no get any authentication wey e come with, so anything wey reach that port fit use your model and read your prompts.

FAQ

Qwen 3.8 27B model dey for Ollama?

No. As of 4 August 2026, the Ollama library no get qwen3.8 namespace. The 27B tags wey dey available na qwen3.5:27b and qwen3.6:27b, and both of dem na Q4_K_M builds of one 27.8 billion parameter dense model. That 3.8 wey you see for search na likely the 27.8B parameter count wey you mistake as version number. Check https://ollama.com/library/qwen3.6/tags for the current list, and run ollama pull qwen3.6:27b` if you want the newest 27B release. Any tag wey no exist go fail with Error: pull model manifest: file does not exist`.

How much RAM I need to run Qwen 27B model for VPS?

32 GB na the minimum wey you go need for Q4_K_M. The weights size na 17 GB, the operating system go take about 1.5 GB, and the KV cache go add roughly 1 GB for every 4000 tokens of context for f16. A 16 GB plan no fit hold the weights at all, and swap no go help because the file dey memory-mapped and the kernel go just dey re-read am from disk for every token. 64 GB go give you space for long context or for Q8_0 weights wey be 30 GB.

How many tokens per second 27B model go give me for CPU?

Divide your memory bandwidth by the size of the weights, then take 50 to 70 percent of that result. A two-channel DDR4-3200 VPS get ceiling near 3 tokens per second and e go deliver about 2. A two-channel DDR5-4800 box get ceiling near 4.5 and e go deliver about 3. Server platforms wey get more channels dey look better for paper, but memory bandwidth dey shared across every tenant for the host, so measure your own with ollama run qwen3.6:27b --verbose and check the eval rate line.

I suppose use Q4 or Q8 for CPU-only VPS?

Use Q4_K_M for almost every case. Q8_0 na 30 GB compared to 17 GB, so e need 64 GB plan and e dey move almost twice the memory per token, wey go cut your tokens per second by half. The quality difference between Q4_K_M and Q8_0 for 27B model no too much for most tasks. Use the extra RAM for longer context instead, because that one go change wetin the model fit do, pass just how e dey arrange im words.

When renting GPU dey cheaper pass big RAM VPS?

If your work no too plenty or if person dey wait for result. A GPU wey get 24 GB memory fit reach roughly 59 tokens per second for these weights, compared to 2 or 3 for normal VPS, and e dey bill only for the hours wey e run. A 64 GB VPS dey bill for the whole month whether the model dey loaded or not. Calculate how many hours per day you really dey generate tokens. If na under two or three hours, hourly GPU rental usually better for speed and cost. If you get continuous low-priority batch work, na the always-on VPS go win.