Qwen 3.8 27B fit run for VPS with Ollama?

You fit run Qwen 3.8 27B for VPS wey no get GPU?

To run Qwen 3.8 27B for VPS, you first need model tag wey really dey exist. As of 4 August 2026, Ollama library no get any qwen3.8 entry at all. The nearest released 27B tag na qwen3.6:27b: e get 27.8 billion parameters, Q4_K_M quantisation, and Apache 2.0 licence. Every command and number below use that tag for Ollama v0.32.5, wey dem publish 27 July 2026.

The short answer na yes for VPS wey get 32 GB or more, but e go slow. A 27B dense model for Q4 need about 17 GB RAM for weights alone, before e store even one context token. That one completely rule out 8 GB and 16 GB plans. For common two-channel DDR4 VPS, the ceiling dey around 3 tokens per second, and that one slow pass how most people dey read.

Where 3.8 come from? Most likely na the parameter count. Ollama page for qwen3.6:27b report 27.8B parameters, and e easy to remember 27.8 later as 3.8. There also dey qwen3.5:27b, wey be the same Q4_K_M build from the previous release. Check the live list before you copy any command, for Ollama qwen3.6 tag page. If real qwen3.8 release later, the calculation here still apply, because e depend on parameter count and bits per weight, no be the version number.

Which Ollama tag to pull, and how to check

If you pull tag wey no dey exist, e go show clear error. So, you fit settle this quickly for the box itself.

ollama pull qwen3.8:27b
# Error: pull model manifest: file does not exist

ollama pull qwen3.6:27b
ollama show qwen3.6:27b

ollama show dey print the architecture, parameter count, context length, and quantisation for the tag wey you actually get. If the parameter line read 27.8B and the quantisation line read Q4_K_M, you get the build wey this guide dey use. The library still get qwen3.6:27b-q8_0 and qwen3.6:27b-bf16 for the same weights with higher precision. E also get some 35b-a3b tags wey be MoE (mixture of experts) models, and dem dey behave very differently for CPU. We go talk more about dem below.

Parameter count times bytes per weight

[
  {
    "label": "Q4_K_M",
    "size_gb": 17,
    "bits_per_weight": 4.89,
    "notes": "published tag qwen3.6:27b"
  },
  {
    "label": "Q5_K_M",
    "size_gb": 19.8,
    "bits_per_weight": 5.7,
    "notes": "computed, no library tag exists"
  },
  {
    "label": "NVFP4",
    "size_gb": 20,
    "bits_per_weight": 5.76,
    "notes": "published tag 27b-nvfp4"
  },
  {
    "label": "Q8_0",
    "size_gb": 30,
    "bits_per_weight": 8.63,
    "notes": "published tag 27b-q8_0"
  },
  {
    "label": "BF16",
    "size_gb": 56,
    "bits_per_weight": 16.1,
    "notes": "published tag 27b-bf16"
  }
]

Formula na one line. Bytes of weights = parameters * bits per weight / 8. For clean 4 bits, 27.8 billion parameters go be 13.9 GB. The shipped Q4_K_M tag na 17 GB, wey mean say e get 4.89 bits per weight for real use.

That difference no be error. K-quant formats no dey store every tensor with the nominal width. Tensors wey compression fit reduce their quality pass dey keep for 5 or 6 bits, and dem normally leave token embedding and output layers for Q6_K or Q8_0. The name for the format na average, and the average dey near 4.9. Same thing dey happen for the other end: 56 GB for BF16 na 16.1 bits per weight instead of fixed 16, because the file still carry metadata and full-precision embedding table.

Q5_K_M no get published tag for this model, so the 19.8 GB row na calculation with the usual 5.7 bits per weight for that format, instead of measured value. Q8_0 nearly double Q4 reach 30 GB. For CPU-only box, that doubling dey cost twice the memory traffic per token, so e also roughly cut your tokens per second by half. Q4_K_M na the correct default here for that reason alone.

Wetin KV cache cost as context dey grow

Weights na fixed cost. KV cache (key and value cache, attention state wey model dey keep for every token wey e don see) dey grow straight with context length, and na where most people actually run out of RAM.

[
  {
    "label": "4k tokens",
    "kv_f16_gb": 1,
    "kv_q8_gb": 0.5
  },
  {
    "label": "8k tokens",
    "kv_f16_gb": 2,
    "kv_q8_gb": 1
  },
  {
    "label": "16k tokens",
    "kv_f16_gb": 4,
    "kv_q8_gb": 2
  },
  {
    "label": "32k tokens",
    "kv_f16_gb": 8,
    "kv_q8_gb": 4
  },
  {
    "label": "64k tokens",
    "kv_f16_gb": 16,
    "kv_q8_gb": 8
  },
  {
    "label": "128k tokens",
    "kv_f16_gb": 32,
    "kv_q8_gb": 16
  }
]

Those figures assume the shape Qwen don use for recent dense models for this size class: 64 layers, 8 key/value heads under GQA (grouped-query attention), and head dimension of 128. That one come to 256 KiB per token for f16, so 8 GB for 32k tokens and 32 GB for 128k. No use my arithmetic take replace wetin dey for your own box. Load the model and read the SIZE column of ollama ps, wey dey report weights plus cache plus overhead as one figure.

Na why the 256K context for model card be headline, no be plan. If you fill am for f16, e go cost 64 GB cache on top of the weights, for machine wey don already spend 17 GB on weights. Ollama no dey give you the full window by default. E dey load much smaller one, and you go raise am deliberately with OLLAMA_CONTEXT_LENGTH. Raise am step by step and check ollama ps after each change.

Two settings fit cut the cache reach half or better. OLLAMA_KV_CACHE_TYPE=q8_0 dey store the cache for 8 bits instead of 16, so 32k tokens go reduce from 8 GB to 4 GB. E need flash attention, so set OLLAMA_FLASH_ATTENTION=1 too, and confirm the drop for ollama ps instead of assuming say e apply. OLLAMA_NUM_PARALLEL=1 matter just as much. Ollama fit serve several requests at once, and each slot get e own slice of context, so if you leave parallelism for default, e go quietly multiply the cache wey you budget for. If more than one person go use this box, na that multiplication dey start the wahala, and the number of concurrent users wey self-hosted model fit serve dey depend on cache slots and queue depth long before core count.

Wetín fit enter 8, 16, 32 and 64 GB of RAM

[
  {
    "label": "8 GB",
    "q4_max_ctx_ktok": 0,
    "q8_max_ctx_ktok": 0,
    "notes": "Weights alone exceed the box. Use a 4b or 8b model."
  },
  {
    "label": "16 GB",
    "q4_max_ctx_ktok": 0,
    "q8_max_ctx_ktok": 0,
    "notes": "17 GB of weights does not fit in 16 GB of RAM."
  },
  {
    "label": "32 GB",
    "q4_max_ctx_ktok": 32,
    "q8_max_ctx_ktok": 0,
    "notes": "Q4 fits with room to spare. Q8 weights do not fit."
  },
  {
    "label": "64 GB",
    "q4_max_ctx_ktok": 128,
    "q8_max_ctx_ktok": 64,
    "notes": "Both fit. Q8 leaves much less room for context."
  }
]

Read the two numbers as thousands of context tokens wey fit stay alongside the weights, for f16 cache, on a headless Linux VPS wey get about 1.5 GB remain for the operating system plus small extra margin. Zero mean say the weights themselves no fit, so nothing fit enter.

8 GB and 16 GB no be close call. 17 GB of weights no fit inside 16 GB of RAM, and no context setting fit change that. Adding swap no go solve am too. Ollama dey memory-map the GGUF file. So when the resident pages pass RAM, the kernel start evicting and reading dem again. Then every token dey pull gigabytes from disk. The box go dey high iowait and produce far below one token per second.

32 GB na the entry point. Weights take 17 GB, and roughly 13 GB remain. This fit cover about 32k tokens of f16 context with some margin. Q8_0 weights at 30 GB no fit for this tier at all.

64 GB dey comfortable. Q4 leave space for around 128k tokens of context, and Q8_0 weights fit with about 64k tokens behind dem. Before you pay for 64 GB to get Q8, understand wetin you dey buy: output wey better small, but speed wey reduce by half, for machine wey already slow. For almost everybody, Q4 with longer context na better trade.

How fast CPU inference dey for VPS?

To generate one token from a dense model, the system must read every weight from memory one time. No be some of dem. Na all of dem. So, the speed limit no be your core count. Na memory bandwidth divided by weight size. For Q4, that one mean 17 GB of memory traffic for each token.

[
  {
    "label": "DDR4-2666, 2 channel",
    "mem_bandwidth_gb_s": 42.6,
    "ceiling_tok_s": 2.5
  },
  {
    "label": "DDR4-3200, 2 channel",
    "mem_bandwidth_gb_s": 51.2,
    "ceiling_tok_s": 3
  },
  {
    "label": "DDR5-4800, 2 channel",
    "mem_bandwidth_gb_s": 76.8,
    "ceiling_tok_s": 4.5
  },
  {
    "label": "DDR4-3200, 8 channel",
    "mem_bandwidth_gb_s": 204.8,
    "ceiling_tok_s": 12
  },
  {
    "label": "DDR5-4800, 12 channel",
    "mem_bandwidth_gb_s": 460.8,
    "ceiling_tok_s": 27.1
  }
]

Those ones na ceiling, no be measurements. Real output normally reach about 50 to 70 percent of the figure wey dey show, because memory latency and imperfect prefetching mean say you no go ever reach the theoretical peak. VPS wey get two-channel DDR4-3200 get ceiling of 3 tokens per second, so expect about 2. Machine wey get two-channel DDR5-4800 get ceiling of 4.5, so expect about 3.

The large server rows come with one warning. Twelve-channel EPYC platform get 460.8 GB/s and ceiling of 27.1 tokens per second, but you no dey rent complete EPYC. Memory bandwidth na resource for the whole host, and every tenant for that machine dey share am. So, 8 vCPU slice no come with twelve channels of exclusive bandwidth. GPU-focused guides dey skip this matter completely. Na this reason two VPS plans wey get the same vCPU count fit differ by factor of three for the same model.

More vCPUs stop to help early for the same reason. Once the cores request data faster than the memory controller fit deliver am, extra threads just add scheduling overhead and nothing more. Set OLLAMA_NUM_THREAD to your physical core count, measure am, then try half that number. For many shared plans, the lower setting dey faster.

Prompt processing dey behave differently. Prefill, wey be the pass over your input before the first token appear, depend more on compute than bandwidth, so e dey scale with cores. The practical result na long pause before output start for large prompt, followed by the slow steady rate wey we explain above. Use --verbose to time both halves separately. E dey print one prompt eval rate and one eval rate for every request.

If dense 27B model too slow, check the qwen3.6:35b-a3b tags before you give up on CPU. Those tags activate about 3 billion parameters for each token instead of all 27.8 billion. So, memory traffic for each token drop by almost one order of magnitude, even though the file for disk bigger. You dey trade RAM footprint for speed. Runtime choice matter here too, and Ollama and llama.cpp expose different CPU tuning controls for the same underlying inference code.

When to rent a GPU hour instead

[
  {
    "label": "L40S, 48 GB",
    "mem_bandwidth_gb_s": 864,
    "ceiling_tok_s": 51
  },
  {
    "label": "RTX 4090, 24 GB",
    "mem_bandwidth_gb_s": 1008,
    "ceiling_tok_s": 59
  },
  {
    "label": "A100, 80 GB",
    "mem_bandwidth_gb_s": 2039,
    "ceiling_tok_s": 120
  },
  {
    "label": "H100 SXM, 80 GB",
    "mem_bandwidth_gb_s": 3350,
    "ceiling_tok_s": 197
  }
]

The same formula wey you apply to published GPU memory bandwidth dey give different kind answer. A 24 GB consumer card get ceiling of 59 tokens per second for these weights. A current data centre card fit reach 197. You no fit close this gap by tuning thread counts. The card dey run memory at 1008 GB/s, while your VPS dey run for tens.

So use workload take draw the line, no be personal preference. CPU inference na the correct choice when the work dey asynchronous and nobody dey wait for am: overnight summarisation of plenty documents, or nightly classification job wey go run while you dey sleep. Rent GPU immediately when person dey wait for output, or when requests dey come faster than one every 30 seconds, because CPU-only box no get batching headroom and queue go just dey grow.

The cost comparison no dey as obvious as e look. 64 GB VPS go bill you every hour of the month whether model dey loaded or not, while GPU instance go bill only for the hours wey you keep am running. If your real usage na two hours per day, rented GPU fit be both faster and cheaper. First calculate your duty cycle, then price am. Picking a VPS with a GPU cover wetin you need check for the instance itself, and vLLM pulls ahead of Ollama once you serve concurrent requests on a GPU because e dey batch dem properly.

People dey forget one third option. Keep the 27B for CPU batch work, then put hosted API model in front of the interactive path. Nothing force you make one model serve both.

Install Ollama and measure your own box

The install script na the official one, and e go set up a systemd service wey dey run as dedicated ollama user.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
free -g

ollama --version suppose print 0.32.5 or later. Check free -g before you pull anything. If total column for the Mem line show below 32, stop here and choose smaller model, because pulling 17 GB wey you no fit run na waste of one hour and plenty disk space.

Set runtime options inside systemd override instead of for your shell. The model dey run inside the service, so e no go ever see your interactive environment.

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="OLLAMA_KEEP_ALIVE=60m"

sudo systemctl restart ollama
ollama pull qwen3.6:27b
ollama run qwen3.6:27b --verbose "Name two Linux distributions."

The --verbose output na the measurement wey you need. eval rate na your tokens per second during generation. prompt eval rate na your prefill speed. load duration na how long e take read the weights from disk, and na why dem set OLLAMA_KEEP_ALIVE=60m: for CPU, reloading 17 GB from disk for every request dey cost pass the request itself.

While the model dey loaded, check the footprint from second terminal.

ollama ps

The SIZE column na the real memory footprint, including the KV cache, and e suppose dey close to the weights plus the row for your context length for the KV chart. For 8192 tokens with 8-bit cache, expect around one gigabyte on top the weights, compared with 2 GB if the cache remain for f16. The PROCESSOR column suppose show 100% CPU. If e show anything else, something don claim a GPU, and the speed numbers for this guide no describe your box.

Failure modes and the exact strings you will see

The model refuses to load. Ollama go print one line wey name both figures, for the shape model requires more system memory (18.6 GiB) than is available (15.2 GiB). Na the good kind failure be this, because Ollama check am before e allocate memory, instead make kernel handle am by force. Reduce the context length, use smaller tag, or move go bigger plan.

The process disappears in the middle of an answer. Client no show anything useful, and journalctl -u ollama -n 50 show say the service dey restart. Run dmesg -T | tail, and line wey read Out of memory: Killed process ... (ollama) mean say kernel OOM killer don stop am. This one happen when pre-load check pass but cache grow pass the estimate during long conversation. Reduce the context length.

The pull fails at once. Error: pull model manifest: file does not exist mean say the tag no dey for the library. If you type qwen3.8:27b, e go produce exactly this result, and any mistake for the version number go do the same thing. Confirm the tag for the library page before you blame your network.

Everything works but it is unbearably slow. If e dey below one token per second for machine wey get enough RAM, that one point to paging, no be compute problem. Run vmstat 1 while e dey generate. Non-zero value for si or so column mean say kernel dey swap, and the fix na less context or fewer loaded models. Steady high wa with no swap activity mean say system dey read the memory-mapped weights again from disk, which mean say dem no really fit for memory.

The first token takes 30 seconds and then output speeds up. Na prefill be that, and e normal. Long system prompt dey cost processing time for every request wey no hit the cache, so shorten the system prompt before you tune anything else.

CPU-only 27B dey actually good for wetin

Set your expectation based on the numbers, no be hope. For two to four tokens per second, 500-token answer go take between two and four minutes. This no usable for chat, but e work well for queue. Document summarisation, bulk tagging, field extraction from backlog of files, and unattended code review fit tolerate am, because nobody dey wait for the reply. Coding assistance dey exactly for that boundary. So pointing coding agent to model wey you host fit pay for background jobs like commit messages and test scaffolding, but e no good for inline suggestions wey you go sit down dey wait for.

Privacy na the real argument. The model dey run for hardware wey you rent and control. No request comot from the box, and no per-token bill dey. This worth plenty for regulated data, even at three tokens per second. Compare am honestly with the alternative: self-hosting frontier-scale model need order of magnitude more hardware, and 27B for CPU na the cheapest point for that curve where the output still worth reading.

If na your first Ollama install, the full walkthrough for running Ollama on VPS cover the service setup, HTTP API, and firewall rules wey this guide assume say you don already get. No expose port 11434 to internet. Ollama ships without authentication of its own, so anything wey reach the port fit use your model and read your prompts.

FAQ

Qwen 3.8 27B model dey for Ollama?

No. As of 4 August 2026, Ollama library no get qwen3.8 namespace. The 27B tags wey dey exist na qwen3.5:27b and qwen3.6:27b, and both na Q4_K_M builds of 27.8 billion parameter dense model. The 3.8 for the search term almost certainly be the 27.8B parameter count wey person remember as version number. Check https://ollama.com/library/qwen3.6/tags for the current list, and pull qwen3.6:27b if you want the newest released 27B. Tag wey no exist go fail with Error: pull model manifest: file does not exist.

How much RAM I need to run Qwen 27B model for VPS?

32 GB na the practical minimum for Q4_K_M. The weights na 17 GB, the operating system need around 1.5 GB, and KV cache add roughly 1 GB for every 4000 tokens of context at f16. 16 GB plan no fit hold the weights at all, and swap no go help because the file dey memory-mapped and kernel just dey read am again from disk for every token. 64 GB give you space for long context or Q8_0 weights at 30 GB.

How many tokens per second 27B model go give me for CPU?

Divide your memory bandwidth by the size of the weights, then take 50 to 70 percent of the result. Two-channel DDR4-3200 VPS get ceiling near 3 tokens per second and deliver about 2. Two-channel DDR5-4800 box get ceiling near 4.5 and deliver about 3. Server platforms with more channels look much better on paper, but memory bandwidth dey shared across every tenant for the host, so measure your own with ollama run qwen3.6:27b --verbose and read the eval rate line.

I suppose use Q4 or Q8 for CPU-only VPS?

Q4_K_M, for almost every case. Q8_0 na 30 GB compared with 17 GB, so e need 64 GB plan and e move almost twice as much memory per token, which reduce your tokens per second by roughly half. The quality difference between Q4_K_M and Q8_0 for 27B model small for most tasks. Use the RAM for longer context instead, because that one change wetin the model fit do, rather than how e phrase things.

When renting GPU go cheaper than big RAM VPS?

When your duty cycle low or person dey wait. GPU with 24 GB memory reach roughly 59 tokens per second on these weights, compared with 2 or 3 for typical VPS, and e bill only for the hours wey e run. 64 GB VPS bill for the whole month whether the model load or not. Calculate how many hours per day you really generate tokens. If na under two or three hours, hourly GPU rental usually win for both speed and cost. Continuous low-priority batch work na where always-on VPS win.