Ollama vs llama.cpp: Which One Fit Your VPS?

Ollama vs llama.cpp: ki layer you wan run?

Ollama and llama.cpp no be competitors the way this question imply. llama.cpp na the inference engine: e dey load model file and turn prompt into tokens. Ollama na model manager, background daemon, and HTTP API wey dey sit on top of that engine. Ollama README still list llama.cpp as its inference backend (checked 2 August 2026). So the real question na which layer you wan operate for your VPS, no be which one fast pass.

Run Ollama when you want service wey go fetch models by name and continue to work without attention. Run llama.cpp directly when the box small and you need choose the exact model file, exact context size, and exact thread count, because for small VPS every one of these settings dey use memory wey you no get.

Wetin each project really be

llama.cpp na C and C++ implementation for transformer inference wey build on ggml library. E dey read GGUF files. GGUF (GGML universal file format) na single-file container wey hold the weights, the tokeniser, and the metadata wey engine need to run the model. The project dey release separate binaries for separate jobs. llama-server na HTTP server, llama-cli na interactive prompt, and llama-bench dey measure throughput. Releases get build number tags instead of semantic version. The current tag na b10224, wey dem publish on 2 August 2026, and new tag dey land for most working days.

Ollama na Go program. Background daemon, wey ollama serve start, dey load models and answer HTTP requests, while command line client dey talk to the daemon. Behind both of dem, registry for ollama.com dey hold prepacked models. Ollama dey use semantic versions, and v0.32.5 ship on 27 July 2026. ollama pull dey fetch GGUF together with prompt template and set of default parameters, then store am under /usr/share/ollama/.ollama/models for Linux.

Na this packaging be the main difference. Ollama dey decide the quantisation, template, and context length for you, and give you one name to remember. llama.cpp no dey decide anything and e give you flags.

Axis 1: control of model and quantisation

Quantisation dey reduce each weight from 16 or 32 bits go 4, 5, or 8 bits. Na this one make model wey get 8 billion parameters fit enter the RAM of normal VPS. Once you sabi the pattern, GGUF naming easy to read: Q4_K_M mean 4-bit K-quant, medium size. Higher number dey keep more precision and dey use more memory.

[
  {
    "label": "Q2_K",
    "file_size_gib": 2.96
  },
  {
    "label": "Q3_K_M",
    "file_size_gib": 3.74
  },
  {
    "label": "Q4_K_M",
    "file_size_gib": 4.58
  },
  {
    "label": "Q5_K_M",
    "file_size_gib": 5.34
  },
  {
    "label": "Q6_K",
    "file_size_gib": 6.14
  },
  {
    "label": "Q8_0",
    "file_size_gib": 7.95
  }
]

Na the file sizes wey dem publish for bartowski/Meta-Llama-3.1-8B-Instruct-GGUF repository for Hugging Face, read on 2 August 2026 and convert from bytes to GiB. 6 builds of one model dey there, and the smallest one na 2.96 GiB, compared with 7.95 GiB for the biggest one. The common default, Q4_K_M, na 4.58 GiB. For 4 GiB VPS, na this choice alone go decide whether the model go load at all.

With llama.cpp, you specify the file, so na you go pick that row.

llama-server -m ~/models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf \
  -c 4096 -t 4 --host 127.0.0.1 --port 8080

-c na the context size for tokens, -t na the thread count, and -ngl set how many layers go move to GPU (0 for box wey CPU-only). E no go guess anything for you.

With Ollama, the quantisation dey come with the tag wey you pull, and ollama ls show wetin you actually get for disk. If the registry no get the build wey you want, import GGUF by yourself. Write a Modelfile:

FROM ./Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf
PARAMETER num_ctx 4096

Then build am and check the result:

ollama create llama31-q4 -f ./Modelfile
ollama ls

Context length na the setting wey dey catch people. Ollama dey choose its default from the VRAM wey dey available, and box wey no get GPU dey enter the smallest bucket: 4096 tokens. If you send am document of 20,000 tokens, the extra tokens go drop before the model ever see dem. So the answer fit confidently wrong about file wey e only read halfway. Increase am with OLLAMA_CONTEXT_LENGTH for the daemon, or with PARAMETER num_ctx inside Modelfile. llama.cpp no get any default wey you fit trust either. Set -c explicitly and know wetin you set.

The memory calculation wey nobody dey show you

The model file no be the whole cost. The KV cache (key/value cache) dey keep one entry for every layer and every context token, and e dey grow as the conversation dey longer.

Make we work am out for Llama 3.1 8B. The model get 32 layers, 8 key/value heads, and head dimension of 128. Every token dey store one key and one value, each one dey use 2 bytes for f16, so 2 x 8 x 128 x 2 = 4096 bytes for each layer. Across 32 layers, na 128 KiB for every token. Context of 4096 tokens therefore dey use 512 MiB, while context of 32,768 tokens dey use 4 GiB.

So, Q4_K_M 8B model for 4k context need roughly 4.58 GiB for weights, plus about 0.5 GiB for cache, plus the runtime itself. E no go fit inside 4 GiB RAM. E go fit for 8 GiB, with space to work. If you raise the context to 32k for that same 8 GiB machine, the cache alone go chop all the remaining memory. Monitor am live with free -h while the model dey loaded, and no trust estimate wey you never measure. If you dey size system for something well pass 8B, this same calculation for 27B model for CPU-only VPS show wetin each tier from 8 to 64 GB fit really hold.

Ollama dey multiply this effect. OLLAMA_NUM_PARALLEL default na 1, and the memory wey model need dey scale with that number multiplied by the context length. If you raise both at once, the daemon go quietly request RAM wey be several times more than wetin you expect. This same calculation set the limit for simultaneous users, because every concurrent request need its own part of KV cache. Na this be why server wey dey okay for one person go stall when five people dey use am.

Axis 2: daemon wey you need operate

The Ollama install script dey write systemd unit, create one ollama system user, and enable the service. You get lifecycle management without writing any of am yourself. Configuration dey pass through systemd:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=8192"
Environment="OLLAMA_KEEP_ALIVE=30m"

sudo systemctl daemon-reload
sudo systemctl restart ollama
journalctl -e -u ollama

OLLAMA_KEEP_ALIVE dey matter more for CPU VPS than anywhere else. Models dey stay for memory for 5 minutes by default, then dem go unload. The next request must read the whole file from disk again before e fit answer, so 4.58 GiB reload fit turn two-second reply to thirty-second one for slow storage. Long keep-alive fit solve the latency, but e go permanently use the RAM. Both na real costs. Choose the one wey go hurt less.

llama.cpp no provide daemon, so you go write the unit yourself as /etc/systemd/system/llama-server.service:

[Unit]
Description=llama.cpp server
After=network-online.target

[Service]
ExecStart=/usr/local/bin/llama-server -m /srv/models/model-Q4_K_M.gguf -c 4096 -t 4 --host 127.0.0.1 --port 8080
Restart=always
RestartSec=3
User=llama

[Install]
WantedBy=multi-user.target

Enable am with sudo systemctl enable --now llama-server. The process go hold the model for the whole time wey e dey run. Nothing go unload when e idle, so you no go get unexpected reload, but you no fit reclaim the memory unless you stop the service. If writing units dey new to you, na the same pattern as running your own services under systemd for VPS.

Axis 3: the API wey your app go talk to

This axis don narrow plenty. Both projects now dey speak OpenAI chat format, so most client libraries fit work with either one after you only change base URL.

Ollama dey listen for 127.0.0.1:11434. Its OpenAI-compatible route na http://localhost:11434/v1/chat/completions, and e still get native API for /api/chat alongside am. Dem don document Anthropic-compatible route too.

curl -X POST http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "llama31-q4", "messages": [{"role": "user", "content": "Say this is a test"}]}'

llama-server dey listen for 127.0.0.1:8080 and e dey serve /v1/chat/completions, /v1/completions and /v1/embeddings, plus e own /completion endpoint and built-in web UI. E still expose operational routes wey Ollama no get: /health for readiness probe, /props for the settings of the loaded model, /slots for wetin each request slot dey do, and /metrics for Prometheus format. If you plan monitor this service, na this difference fit decide am.

Neither server go switch on authentication for you. Both dey default to loopback for good reason. Reach dem through SSH tunnel or from behind reverse proxy, and never open 11434 or 8080 to internet.

Wetin CPU-only VPS fit honestly do

CPU-only VPS dey run small models slowly. Na the honest summary be that, and the useful part na to know where the limit dey. Measure first before you design anything around am:

llama-bench -m ~/models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf -p 512 -n 128

The pp column na prompt processing speed, while the tg column na token generation speed. Both dey measured for tokens per second. For shared vCPU plan, 8B model with Q4_K_M usually dey reach low single digits for tg. Prompt processing na the part wey dey pain: the whole prompt must process before the first output token show, so long system prompt dey add waiting time to every request.

CPU fit handle: 1B to 4B model for classification, extraction, short summaries, or routing. Replies go arrive within seconds, and the memory go fit inside normal plan. CPU no fit handle well: interactive chat at reading speed, coding assistants, long-document work, or anything wey get agent loop making plenty calls one after another. Loop wey make twelve calls at four seconds each go take one minute before e produce anything. If coding assistant na the plan from the beginning, pointing agent to model wey you host by yourself explain which jobs small local model genuinely dey do better and which ones must remain on hosted API.

Two options dey when the numbers no work. If na concurrency be the problem, meaning many users dey hit one model at the same time, engine choice go change, and comparison of Ollama and vLLM for concurrent serving cover that matter. If na raw speed be the problem, answer na VPS wey get GPU attached, where -ngl start to mean something. Before either option, get baseline for the hardware itself, because disk and memory bandwidth affect load time almost as much as CPU. Repeatable VPS benchmark worth the one hour.

Install llama.cpp, pin one build

Projects wey dey move every week, so record the version wey you deploy. The upstream one-liner dey install the current build:

curl -LsSf https://llama.app/install.sh | sh
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

To pin one specific build, use the prebuilt tarball from the releases page instead. Build b10224 na the current tag as of 2 August 2026:

curl -LO https://github.com/ggml-org/llama.cpp/releases/download/b10224/llama-b10224-bin-ubuntu-x64.tar.gz
tar xf llama-b10224-bin-ubuntu-x64.tar.gz
find . -type f -name 'llama-server'

Or build that same tag from source:

sudo apt update && sudo apt install -y build-essential cmake git libssl-dev
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout b10224
cmake -B build
cmake --build build --config Release -j $(nproc)

libssl-dev na the documented dependency for the HTTPS features. The compile fit take several minutes and e need more RAM than the smallest plans get, so build am for a bigger machine and copy the binaries if the small one run out of memory.

Install Ollama, pin am to one version

curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.32.5 sh
ollama -v

The script dey read OLLAMA_VERSION, so you fit hold one release wey you don confirm say e good, instead of taking any version wey dem release this morning. Dem publish v0.32.5 on 27 July 2026. Manual option dey too if you no wan pipe script enter shell:

sudo rm -rf /usr/lib/ollama
curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst | sudo tar x -C /usr
ollama -v

Manual method no dey create the systemd unit or service user, so na you go add dem yourself. The complete Ollama for VPS guide explain how to set up the service step by step.

Failure modes, and the strings wey you go see

Ollama no gree load the model. ollama run dey return line wey get this shape:

Error: model requires more system memory (5.6 GiB) than is available (3.2 GiB)

Ollama dey check the size before e load am, so e fail quick and tell you why. Move down one quantisation row, reduce the context length, or choose smaller model.

llama.cpp no fail, e just dey crawl. llama.cpp dey memory-map the GGUF by default, so file wey big pass RAM still fit start. Kernel go dey page weights in and out from disk for every token, and generation go drop reach seconds per token while disk dey pinned for 100 percent. Pass --no-mmap to force real allocation, make e fail immediately instead of degrading. When kernel step in, dmesg go show the reason:

Out of memory: Killed process 1234 (llama-server)

The model file no go load at all. GGUF wey you build for model family newer than your engine go return error wey name the architecture wey e no know:

error loading model architecture: unknown model architecture: 'qwen3next'

The fix na engine upgrade, no be different file. Na this be the price of pinning, and na why you write down the build number. You need know the version wey you dey upgrade from.

The API dey answer locally but e no answer from your app. Ollama dey bind 127.0.0.1:11434, so another host go get connection refused. Set OLLAMA_HOST=0.0.0.0:11434 through systemctl edit ollama only when the port dey behind firewall or private network, because the API no get authentication in front of am.

The first reply after pause dey very slow. The 5 minute idle unload don happen, and the model dey read from disk again. ollama ps wey you run just before the request go show say nothing dey loaded, and this confirm am. Increase OLLAMA_KEEP_ALIVE.

So which one you suppose run?

Run Ollama when you want make e manage models for you and give you OpenAI-shaped endpoint without extra work. E dey make sense as default for first deployment, and for anything wey model choice go dey change often.

Run llama.cpp directly when memory tight reach where you need choose the quantisation row by yourself, when you want /health, /slots and /metrics for monitoring, or when you need flag wey Ollama no expose. Na the honest choice for VPS where model barely fit, because na the same settings wey make am fit be the ones Ollama dey choose for you.

Running both na normal. Use Ollama for experiments, and llama.cpp for the one model wey you put for production and no wan make e change.

FAQ

Ollama na just wrapper around llama.cpp?

E close, but the wrapper dey do real work. Ollama README list llama.cpp as the inference backend (checked 2 August 2026). On top of am, Ollama add model registry, prompt template wey dey turn chat messages into prompt, default sampling parameters, daemon wey fit unload model when e idle, and HTTP API. When you compare tokens per second with the same settings, na the same engine you dey compare with itself. The real choice na the management layer.

Which one dey faster for CPU-only VPS?

Dem share the same engine, so if model file, quantisation, context size, and thread count dey the same, dem go perform almost alike. Differences wey people report usually come from different defaults, especially context length and thread count, no be from the engine itself. Use llama-bench -m <file> -p 512 -n 128 measure am, then compare the tg column for your own box before you trust any published figure.

I fit use my own GGUF file with Ollama?

Yes. Put the file for the server, write a Modelfile wey first line na FROM ./your-model.gguf, add any PARAMETER lines wey you need, like num_ctx, then run ollama create your-name -f ./Modelfile. ollama ls go list am together with anything wey you pull from the registry. Na so you fit use quantisation wey the registry no carry.

How much RAM I need for 8B model?

Plan for the file size, plus KV cache, plus the runtime. Q4_K_M build of Llama 3.1 8B dey about 4.58 GiB for disk, and 4096 token context dey add roughly 512 MiB cache. So 8 GiB RAM dey comfortable, but 4 GiB no reach. The cache dey grow with context: the same model with 32,768 token context need about 4 GiB cache by itself. With Ollama, remember say the requirement also dey scale with OLLAMA_NUM_PARALLEL.