Ollama vs llama.cpp: Which One Fit Your VPS?

Ollama vs llama.cpp: which layer you want run?

Ollama and llama.cpp no be competitors the way the question take imply. llama.cpp na the inference engine: e dey load model file and turn prompt to tokens. Ollama na model manager, background daemon, and HTTP API wey dey sit on top that engine. Ollama README still list llama.cpp as its inference backend (checked 2 August 2026). So the real question na which layer you want operate for your VPS, no be which one fast pass.

Run Ollama when you want service wey go fetch models by name and continue to work without attention. Run llama.cpp directly when the box small and you need choose the exact model file, exact context size, and exact thread count, because for small VPS every one of those settings dey use memory wey you no get.

Wetin each project actually be

llama.cpp na C and C++ implementation for transformer inference wey dey use ggml library. E dey read GGUF files. GGUF (GGML universal file format) na single-file container wey hold the weights, the tokeniser, and the metadata wey engine need to run the model. The project dey release separate binaries for separate work. llama-server na HTTP server, llama-cli na interactive prompt, and llama-bench dey measure throughput. Releases dey use build number as tag instead of semantic version. The current tag na b10224, dem publish am on 2 August 2026, and new tag dey land for most working days.

Ollama na Go program. Background daemon, wey ollama serve start, dey load models and answer HTTP requests, while command line client dey talk to that daemon. Behind both of dem, registry for ollama.com dey hold prepacked models. Ollama dey use semantic versions, and v0.32.5 ship on 27 July 2026. ollama pull dey fetch one GGUF together with prompt template and set of default parameters, then store am under /usr/share/ollama/.ollama/models for Linux. Those files dey sit for root disk and each one fit reach several gigabytes, so if na VPS wey get 25 GB root volume, e good make you know wetin pull leave behind and how to move the model directory go another place before the third download fill am.

Na that packaging be the main difference. Ollama dey choose the quantisation, the template, and the context length for you, and e give you one name to remember. llama.cpp no dey decide anything and e give you flags.

Axis 1: model and quantisation control

Quantisation dey reduce each weight from 16 or 32 bits go 4, 5 or 8 bits. Na wetin make 8 billion parameter model fit inside RAM of ordinary VPS. GGUF naming easy to understand once you sabi the pattern: Q4_K_M mean 4-bit K-quant with medium size. Higher number dey keep more precision and dey use more memory.

[
  {
    "label": "Q2_K",
    "file_size_gib": 2.96
  },
  {
    "label": "Q3_K_M",
    "file_size_gib": 3.74
  },
  {
    "label": "Q4_K_M",
    "file_size_gib": 4.58
  },
  {
    "label": "Q5_K_M",
    "file_size_gib": 5.34
  },
  {
    "label": "Q6_K",
    "file_size_gib": 6.14
  },
  {
    "label": "Q8_0",
    "file_size_gib": 7.95
  }
]

Na the published file sizes for bartowski/Meta-Llama-3.1-8B-Instruct-GGUF repository on Hugging Face, wey dem read on 2 August 2026 and convert from bytes go GiB. 6 builds of one model dey here, and the smallest one na 2.96 GiB compared with 7.95 GiB for the largest. The common default, Q4_K_M, na 4.58 GiB. For 4 GiB VPS, this one choice decide whether the model go load at all. Size na only half of the decision, because row wey you fit afford no automatically mean say e worth using, and wetin Q4, Q8 and fp16 really cost for answer quality go tell you whether the extra gigabytes buy anything wey you go notice.

With llama.cpp, you name the file, so na you go pick the row yourself.

llama-server -m ~/models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -c 4096 -t 4 --host 127.0.0.1 --port 8080

-c na the context size for tokens, -t na the thread count, and -ngl set how many layers go move to GPU (0 for CPU-only box). Nothing dey guessed for you.

With Ollama, the quantisation dey follow the tag wey you pull, and ollama ls show wetin you actually get for disk. If registry no carry the build wey you want, import GGUF yourself. Write Modelfile:

FROM ./Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf
PARAMETER num_ctx 4096

Then build am and check the result:

ollama create llama31-q4 -f ./Modelfile
ollama ls

Context length na the setting wey dey catch people. Ollama dey pick default based on available VRAM, and box wey no get GPU go enter the smallest bucket: 4096 tokens. If you send am 20,000-token document, the extra tokens go drop before model ever see dem, so the answer fit confidently wrong about file wey e only read halfway. Increase am with OLLAMA_CONTEXT_LENGTH for the daemon, or with PARAMETER num_ctx inside Modelfile. If na only one job need the bigger window, you fit set num_ctx per request instead of for the whole server, so the extra cache no go affect every other work wey daemon dey handle. llama.cpp no get default wey you fit trust either. Set -c explicitly and know wetin you set.

The memory calculation wey nobody dey show you

The model file no be the complete cost. The KV cache (key/value cache) dey keep one entry for each layer and each context token, and e dey grow as conversation dey grow.

Calculate am for Llama 3.1 8B. The model get 32 layers, 8 key/value heads, and head dimension na 128. Each token dey store one key and one value, each one dey use 2 bytes for f16. So, 2 x 8 x 128 x 2 = 4096 bytes for each layer. Across 32 layers, na 128 KiB for each token. Context of 4096 tokens therefore dey use 512 MiB, while context of 32,768 tokens dey use 4 GiB.

So, Q4_K_M 8B model for 4k context need roughly 4.58 GiB for weights, plus about 0.5 GiB for cache, plus the runtime itself. E no go fit inside 4 GiB RAM. E go fit for 8 GiB with some space to work. If you raise the context to 32k for that same 8 GiB machine, the cache alone go finish the available headroom. Monitor am live with free -h while the model dey loaded, and no trust estimate wey you never measure. If you dey size system for something far pass 8B, the same calculation wey dem work through for 27B model for CPU-only VPS shows wetin each tier from 8 to 64 GB fit hold.

Ollama dey multiply this. OLLAMA_NUM_PARALLEL default value na 1, and the memory wey model need dey scale according to that number multiplied by the context length. If you raise both at the same time, the daemon fit quietly request RAM wey be several times more than wetin you expect. That same calculation set your limit for simultaneous users, because each concurrent request need its own part of KV cache. Na this be why server wey dey work fine for one person dey stall when five people use am.

Axis 2: daemon wey you gats operate

Ollama install script dey write a systemd unit, create a ollama system user, and enable the service. You get lifecycle management without writing any of am yourself. Configuration dey pass through systemd:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_KEEP_ALIVE=30m"

sudo systemctl daemon-reload
sudo systemctl restart ollama
journalctl -e -u ollama

OLLAMA_KEEP_ALIVE dey matter more for CPU VPS than for anywhere else. By default, models dey stay for memory for 5 minutes, then dem unload. Next request gats read the complete file back from disk before e fit answer, so 4.58 GiB reload fit turn two-second reply to thirty-second one for slow storage. Long keep-alive go fix the latency, but e go permanently use the RAM. Both na real costs. Pick the one wey go pain less. If you decide say the model suppose simply stay resident, set keep_alive so e go survive idle periods and reboots need only a few lines and e go save you from warming the model by hand every time the box restart.

llama.cpp no get daemon, so you gats write the unit yourself as /etc/systemd/system/llama-server.service:

[Unit]
Description=llama.cpp server
After=network-online.target

[Service]
ExecStart=/usr/local/bin/llama-server -m /srv/models/model-Q4_K_M.gguf -c 4096 -t 4 --host 127.0.0.1 --port 8080
Restart=always
RestartSec=3
User=llama

[Install]
WantedBy=multi-user.target

Enable am with sudo systemctl enable --now llama-server. The process go hold the model for the whole time wey e dey run. Nothing go unload when e idle, so you no go get reload surprise, but you no fit reclaim the memory unless you stop the service. If writing units dey new to you, na the same pattern as running your own services under systemd on a VPS.

Axis 3: na API wey your app go talk to

This axis don narrow well-well. Both projects now dey use OpenAI chat format, so most client libraries go work with either one after you only change base URL.

Ollama dey listen on 127.0.0.1:11434. E OpenAI-compatible route na http://localhost:11434/v1/chat/completions, and e still get native API for /api/chat. Dem don document Anthropic-compatible route too.

curl -X POST http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama31-q4", "messages": [{"role": "user", "content": "Say this is a test"}]}'

llama-server dey listen on 127.0.0.1:8080 and e dey serve /v1/chat/completions, /v1/completions and /v1/embeddings, plus e own /completion endpoint and built-in web UI. E also expose operational routes wey Ollama no get: /health for readiness probe, /props for the loaded model settings, /slots for wetin each request slot dey do, and /metrics for Prometheus format. If you plan to monitor this service, na this difference fit decide am.

Neither server dey enable authentication for you. Both dey use loopback by default for good reason. Reach dem through SSH tunnel or from behind reverse proxy, and never open 11434 or 8080 to internet.

Wetin CPU-only VPS fit honestly do

CPU-only VPS dey run small models slowly. Na the honest summary be that, and the useful part na to know where the limit dey. Measure am before you design anything around am:

llama-bench -m ~/models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf -p 512 -n 128

The pp column na prompt processing speed, while the tg column na token generation speed. Both dey measure tokens per second. For shared vCPU plan, 8B model for Q4_K_M normally dey reach low single digits for tg. Prompt processing na the part wey dey cause delay: the system go process the whole prompt before the first output token show, so long system prompt go add wait to every request. Reply length na the part of the bill wey you fit control, because for three tokens per second, model wey dey ramble for 600 tokens go hold the box for three minutes. So, capping the output with num_predict na the cheapest way to stop one chatty answer from turning into timeout.

CPU fit handle: 1B to 4B model for classification, extraction, short summaries, or routing. Replies go arrive within seconds, and the memory requirement fit normal plan. For worked example with that size instead of size range, Nemotron 3.5 Lightning pulled and measured on a VPS give the exact tag, the RAM e actually need, and the speed e maintain without GPU. CPU no fit handle: interactive chat at reading speed, coding assistants, long-document work, or anything with agent loop wey dey make many calls one after another. Loop wey make twelve calls for four seconds each go take one minute before e produce anything. If coding assistant na still the plan, pointing an agent at a model you host yourself explain which jobs small local model genuinely fit handle better and which ones must remain for hosted API.

There are two ways out when the numbers no work. If na concurrency be the problem, meaning many users dey hit one model at the same time, engine choice go change, and the comparison of Ollama against vLLM for concurrent serving cover that matter. If na raw speed be the problem, answer na a VPS with a GPU attached, where -ngl go start to mean something. Before you choose either option, get baseline for the hardware itself, because disk and memory bandwidth affect load time as much as CPU. A repeatable VPS benchmark worth the one hour.

Install llama.cpp, pin am build

Both project dey change every week, so record the version wey you deploy. The upstream one-liner dey install the current build:

curl -LsSf https://llama.app/install.sh | sh
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

To pin one specific build, use the prebuilt tarball for releases page instead. Build b10224 na the current tag as of 2 August 2026:

curl -LO https://github.com/ggml-org/llama.cpp/releases/download/b10224/llama-b10224-bin-ubuntu-x64.tar.gz
tar xf llama-b10224-bin-ubuntu-x64.tar.gz
find . -type f -name 'llama-server'

Or build that same tag from source:

sudo apt update && sudo apt install -y build-essential cmake git libssl-dev
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout b10224
cmake -B build
cmake --build build --config Release -j $(nproc)

libssl-dev na the documented dependency for the HTTPS features. The compile fit take several minutes and e need more RAM than the smallest plans get, so build am for bigger box and copy the binaries if the small one run out of memory.

Install Ollama, pin am to one version

curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.32.5 sh
ollama -v

The script dey read OLLAMA_VERSION, so you fit hold one release wey you don verify instead of taking anything wey release this morning. v0.32.5 come out on 27 July 2026. Manual option dey too if you no wan pipe script enter shell:

sudo rm -rf /usr/lib/ollama
curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst | sudo tar x -C /usr
ollama -v

The manual method no dey create systemd unit or service user, so na you go add dem. The complete Ollama for VPS walkthrough explain the service setup step by step.

Wetin fit fail, and the strings wey you go see

Ollama no gree load the model. ollama run dey return line wey get this shape:

Error: model requires more system memory (5.6 GiB) than is available (3.2 GiB)

Ollama dey check the size before e load am, so e fail quick and talk why. Move go the next quantisation row, reduce the context length, or choose smaller model.

llama.cpp no fail, e dey crawl. llama.cpp dey memory-map the GGUF by default, so file wey pass RAM still fit start. Kernel go dey page weights in and out from disk for every token, and generation go drop to seconds per token while disk dey pinned at 100 percent. Pass --no-mmap to force real allocation, so e go fail immediately instead of slowing down. When kernel step in, dmesg go show the reason:

Out of memory: Killed process 1234 (llama-server)

The model file no go load at all. GGUF wey dem build for model family newer than your engine go give error wey name the architecture wey e no know:

error loading model architecture: unknown model architecture: 'qwen3next'

The fix na engine upgrade, no be different file. Na the price of pinning, and na why you write down the build number. You need know wetin you dey upgrade from.

The API dey answer locally but e no answer from your app. Ollama dey bind 127.0.0.1:11434, so another host go get connection refused. Set OLLAMA_HOST=0.0.0.0:11434 through systemctl edit ollama only when the port dey behind firewall or private network, because the API no get authentication in front of am.

The first reply after pause dey very slow. The 5 minute idle unload don happen, and the model dey read from disk again. ollama ps wey you run just before the request show say nothing dey loaded, and this confirm am. Increase OLLAMA_KEEP_ALIVE.

So which one you suppose run?

Run Ollama when you want make e manage models for you and provide an OpenAI-shaped endpoint without extra work. E be the correct default for first deployment, and for anything wey model choice go continue to change.

Run llama.cpp directly when memory tight reach where you need choose the quantisation row yourself, when you want /health, /slots and /metrics for monitoring, or when you need a flag wey Ollama no expose. E be the honest choice for VPS where model barely fit, because na the same settings wey make am fit be exactly the ones wey Ollama choose for you.

Running both dey normal. Ollama for experiments, llama.cpp for the one model wey you put for production and never want make e move.

FAQ

Ollama na just wrapper around llama.cpp?

E near, but the wrapper dey do real work. Ollama README list llama.cpp as e inference backend (checked 2 August 2026). Ollama add model registry, prompt template wey dey turn chat messages into prompt, set of default sampling parameters, daemon wey dey unload idle models, and HTTP API on top of am. When you compare tokens per second with the same settings, na the same engine you dey compare with itself. The real choice na the management layer.

Which one fast pass for CPU-only VPS?

Dem share the same engine, so if model file, quantisation, context size, and thread count dey the same, their results go dey close. Differences wey people report usually come from different defaults, especially context length and thread count, instead of the engine itself. Use llama-bench -m <file> -p 512 -n 128 to measure am, then compare the tg column for your own box before you trust any published figure.

I fit use my own GGUF file with Ollama?

Yes. Put the file for the server, write a Modelfile wey e first line be FROM ./your-model.gguf, add any PARAMETER lines wey you need, like num_ctx, then run ollama create your-name -f ./Modelfile. ollama ls go list am beside anything wey you pull from the registry. Na so you fit use quantisation wey the registry no carry.

How much RAM I need for 8B model?

Plan for the file size, plus KV cache, plus runtime. Q4_K_M build of Llama 3.1 8B dey about 4.58 GiB for disk, and 4096 token context add roughly 512 MiB cache. So 8 GiB RAM dey comfortable, but 4 GiB no reach. The cache dey scale with context: the same model with 32,768 token context need about 4 GiB cache by itself. With Ollama, remember say the requirement also dey scale with OLLAMA_NUM_PARALLEL.