Ollama: fix context deadline exceeded

What "context deadline exceeded" actually means

The Ollama error context deadline exceeded is a timeout report. Some piece of Go code set a deadline on the request, the model did not finish inside it, and the deadline expired. Nothing crashed and no file is corrupt. The work was still running when the clock ran out.

The wording comes from Go's standard context package, and that is a useful clue on its own. A Python client built on httpx raises httpx.ReadTimeout instead. A browser shows a plain network error. If you are reading these exact words, a Go program gave up waiting: the Ollama command line tool, the Ollama server itself, or a Go application calling the API (application programming interface).

Five layers can set that deadline. They fail at different points and each one needs a different fix, so the whole job is working out which one fired.

1. Your HTTP client, which gave the request a fixed time budget.
2. The Ollama server's model load timeout, which fires while a large model is being read off disk for the first time.
3. keep_alive, which unloads the model between requests so the next call pays the load cost again.
4. A num_ctx large enough that prompt processing alone runs for minutes on a CPU-only box.
5. A reverse proxy such as nginx or Traefik, cutting the connection before Ollama has replied.

Work down that list in order. Each step below removes one layer from the picture, so you stop guessing.

Reproduce against the API to take the proxy out

Run the request on the server itself, straight at Ollama, with no proxy in between.

time curl -s http://127.0.0.1:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Why is the sky blue?",
  "stream": false
}' | head -c 400

curl sets no overall time limit of its own, only a connect timeout, so this command waits for as long as Ollama needs. That splits the problem in half. If a JSON body comes back, Ollama answered, and the deadline belongs to something in front of it. If this call itself hangs for minutes, the delay is inside Ollama and your proxy is innocent.

Now send the same request through your public URL and time it.

curl -s -o /dev/null -w '%{http_code} %{time_total}\n' \
  -X POST https://llm.example.com/api/generate \
  -d '{"model": "llama3.1:8b", "prompt": "hi", "stream": false}'

A status of 504 printed after a suspiciously round number of seconds, 60.0 or 30.0, is a proxy timeout. Proxies use round defaults. A model does not finish at exactly 60.000 seconds twice in a row. If the direct call is refused instantly rather than slow, you have a listener problem instead of a deadline problem, and which address Ollama binds on port 11434 covers that case.

Watch the server log while the request runs

Open a second session and follow the service log, then send the request again.

journalctl -u ollama --no-pager --follow --pager-end

A healthy cold start logs the model being loaded, then a runner starting, then the request being served. A failed load looks like this instead, and it is the string that identifies the server's own load timeout:

Error: timed out waiting for llama runner to start - progress 0.00 -

That message means the model process never finished starting inside the server's budget. The progress figure tells you how far it got. A value of 0.00 means the runner reported nothing at all before the deadline, which usually means the file is still being read or the machine is swapping. For more detail during the load, restart the service with OLLAMA_DEBUG=1 set and repeat.

Measure whether the delay is loading or generating

Ollama reports its own timings, so you never have to guess this part.

ollama run --verbose llama3.1:8b "Why is the sky blue?"

After the answer it prints total duration, load duration, prompt eval count, prompt eval rate, eval count and eval rate. Run it twice. On the second run load duration should drop to almost nothing, because the model is already resident. If it does not drop, the model is being unloaded between your two runs, which is the keep_alive case further down.

The same numbers come back from the API in the final JSON object, as load_duration, prompt_eval_duration and eval_duration. The documentation states that all durations are returned in nanoseconds, so divide by 10^9 to read seconds.

curl -s http://127.0.0.1:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Why is the sky blue?",
  "stream": false
}' | python3 -c 'import json,sys; d=json.load(sys.stdin); print({k: round(v/1e9, 2) for k, v in d.items() if k.endswith("_duration")})'

Read the biggest number. If load_duration dominates, you have a model load problem, so go to the next two sections. If prompt_eval_duration dominates, prompt processing is the cost, so go to the num_ctx section. If eval_duration dominates, the model is simply generating slowly on this hardware, and no timeout setting will change that. Shorten the output with num_predict, or move to a smaller model.

Raise OLLAMA_LOAD_TIMEOUT, after you check your version

The server variable that governs how long it waits for a model to start is OLLAMA_LOAD_TIMEOUT. Its default has changed between releases, so read it for your build rather than from any article, including this one. Print the version first.

ollama --version

Then open the source for that exact tag, https://github.com/ollama/ollama/blob/<your version>/envconfig/config.go, and search for OLLAMA_LOAD_TIMEOUT. The value in that file is the default your binary compiled in. Set your own value through a systemd drop-in.

sudo systemctl edit ollama.service

Add the variables under a [Service] section, which is the method Ollama's own documentation gives for Linux:

[Service]
Environment="OLLAMA_LOAD_TIMEOUT=15m"
Environment="OLLAMA_KEEP_ALIVE=-1"

sudo systemctl daemon-reload
sudo systemctl restart ollama
systemctl show ollama --property=Environment

The last command prints the environment the service actually received. An empty result means the drop-in was saved outside the editor markers, or under the wrong section name, so nothing you set is in effect. Be clear about what this buys you: a longer load timeout stops the server giving up, and it makes nothing faster. If the model does not fit in memory the machine will swap, the load will crawl, and a bigger number only moves the failure later.

Why the first request after a pause is the slow one

Ollama unloads an idle model to free memory. The keep_alive setting decides when. Ollama's documentation gives the default as 5 minutes, checked in September 2026. So a chat app that is used once an hour reloads the model on every single message, and every message pays the full cold start. The request that times out is the first one after a quiet period, which is exactly the pattern people describe as random.

Check what is resident right now:

ollama ps
curl -s http://127.0.0.1:11434/api/ps

An empty list, or an expiry a few minutes away, confirms it. keep_alive accepts a duration string such as "10m" or "24h", a plain number of seconds, 0 to unload at once, and a negative number to keep the model in memory indefinitely. Set it per request, or set OLLAMA_KEEP_ALIVE on the service for every request.

curl -s http://127.0.0.1:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "keep_alive": -1
}'

A request with a model and no prompt loads the model and returns. That is the documented way to warm a box after a reboot, and it belongs in a small systemd unit so nobody waits for a cold start. The cost is honest: a pinned model holds its memory forever, so on a small box you can pin one model, not four. Keeping a model resident between requests goes through the memory arithmetic and the warm-up unit.

Why a large num_ctx times out before the first token

Before a model writes anything it must read your whole prompt. That stage is prefill, and it is what prompt eval measures. num_ctx sets the context length, which does two things at once. It caps how many tokens the model may consider, and it sizes the KV cache (key value cache) that the server allocates up front. Both grow the work.

On a CPU-only server prefill is slow, and it is linear in the number of prompt tokens. A long document pasted into a chat can spend minutes in prefill while the client sees nothing at all, because streaming has not started yet. The client hits its deadline and reports context deadline exceeded, and the server was working the whole time. Prove it with the numbers from the previous section: run the same prompt with "options": {"num_ctx": 2048} and then with 32768, and compare prompt_eval_duration.

The server default comes from OLLAMA_CONTEXT_LENGTH, and a per-request num_ctx in the options object overrides it. Raising it to the model's advertised maximum because the maximum exists is the usual mistake, since the KV cache allocation can push the model out of RAM and turn a working setup into a swapping one. Choosing num_ctx against your actual memory has the sizing detail.

Why nginx returns 504 Gateway Time-out

nginx documents proxy_read_timeout with a default of 60s, and its error log names the failure plainly:

upstream timed out (110: Connection timed out) while reading response header from upstream

The important detail is in the nginx documentation: the timeout "is set only between two successive read operations, not for the transmission of the whole response". A streaming response resets the clock with every chunk, so streaming chats survive. A request with "stream": false sends nothing until the answer is complete, so the entire generation has to finish inside that one window. This is why the same model works in the chat window and times out from a script.

location / {
    proxy_pass http://127.0.0.1:11434;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_read_timeout 600s;
    proxy_send_timeout 600s;
    proxy_buffering off;
}

sudo nginx -t && sudo systemctl reload nginx

proxy_buffering off matters for streaming. With buffering on, nginx can collect the response and hand it over at the end, so tokens stop appearing one by one and a working stream starts to look like a hang.

Traefik puts the same control on the ServersTransport that the router uses.

http:
  serversTransports:
    ollama:
      forwardingTimeouts:
        dialTimeout: "30s"
        responseHeaderTimeout: "0s"
        idleConnTimeout: "60s"

responseHeaderTimeout covers the wait for response headers after the request is written, and zero means no timeout. The service has to reference the transport by name with serversTransport: ollama, or you have edited a block that nothing uses.

A smaller quantisation loads faster because there is less to read

Quantisation is the precision the weights are stored at. Lower precision means a smaller file, and loading a model is mostly reading that file from disk into memory.

[
  {
    "label": "q4_K_M",
    "download_size_gb": 4.9
  },
  {
    "label": "q8_0",
    "download_size_gb": 8.5
  },
  {
    "label": "fp16",
    "download_size_gb": 16
  }
]

Those are the sizes published on the model page, not measurements from a test box. The default 8B build ships at 4.9 GB. The full precision build of the same model is 16 GB, more than three times as many bytes to read and more than three times the memory to hold. On a rented server with shared storage, that difference is the whole gap between a load that finishes and a load that trips the timeout. Working out which model fits your RAM is the check to run before pulling anything large.

What to change on a rented box

Apply these in the order the measurements point to, one at a time, and re-run the timing command after each one.

1. Pin the model with OLLAMA_KEEP_ALIVE=-1, or warm it at boot, so no user request ever pays the load cost.
2. Drop num_ctx to what your prompts actually need, which shortens prefill and frees the memory the KV cache was holding.
3. Pull a smaller quantisation, so the load reads fewer bytes and the model leaves room for the cache.
4. Raise proxy_read_timeout in nginx or responseHeaderTimeout in Traefik, and turn buffering off so streamed tokens reach the client.
5. Raise the timeout in your own client, since a Go or Python program with a 30 second budget will fail against any model that thinks for longer.

One more cause hides behind all of these. Ollama serves a limited number of requests at once and queues the rest, so a second caller can sit in the queue while its own deadline expires, with no slow model anywhere in sight. The server log shows the request being served late rather than failing. What happens when several people share one Ollama box covers the parallelism settings, and the base install on a VPS covers the service setup these overrides assume.

FAQ

What does "context deadline exceeded" mean in Ollama?

It means a deadline on the request expired before the model answered. The phrase comes from Go's context package, so a Go program printed it: the Ollama command line tool, the Ollama server, or a Go application calling the API. It is a timeout, so nothing is broken or corrupt. The next step is finding which layer set the deadline, because the client, the model load, keep_alive, num_ctx and the reverse proxy all set their own.

Should I raise my client timeout or Ollama's timeout?

Measure first. Send the request with curl on the server itself, straight to http://127.0.0.1:11434, since curl imposes no overall time limit. If that call returns a JSON body, Ollama is answering and the deadline belongs to your client or your proxy, so raise it there. If that call hangs too, the delay is inside Ollama, and the fields load_duration and prompt_eval_duration in the response tell you whether the model is loading or reading your prompt.

Why does the first request time out and the next one work?

Ollama unloads an idle model to free memory, on a schedule set by keep_alive. The documented default is 5 minutes, checked in September 2026. The first request after an idle period reloads the model from disk and pays the full cold start, while a request sent straight after finds it resident and returns quickly. Run ollama ps to see what is loaded and when it expires. Set OLLAMA_KEEP_ALIVE=-1 to keep it in memory, and accept that the memory stays occupied.

Why does it only fail when I go through nginx?

nginx documents proxy_read_timeout with a default of 60s, and that timeout applies between two successive reads rather than to the whole response. A streaming reply resets it with every chunk, while a request sent with "stream": false has to complete inside a single window. That is why the chat window works and a script fails. Look for upstream timed out (110: Connection timed out) while reading response header from upstream in the nginx error log, then raise proxy_read_timeout and set proxy_buffering off.

Does raising OLLAMA_LOAD_TIMEOUT make loading faster?

No. It only changes how long the server waits before it gives up and logs timed out waiting for llama runner to start. If the model does not fit in memory the machine swaps, the load crawls, and a larger timeout moves the failure later without fixing it. Check the default for your build by running ollama --version and reading envconfig/config.go at that tag, then treat a load that needs minutes as a sign to pull a smaller quantisation.