Ollama num_predict: cap the output length

What num_predict does in Ollama

num_predict is the Ollama option that caps how many tokens a model may generate in one response. It counts output tokens only, so the prompt is never charged against it. When the model reaches the cap, generation stops where it is, sometimes mid-word, and the response comes back with done_reason set to length.

That is the whole feature. The difficulty is that Ollama gives you three separate places to set the value, and the setting nearest the request wins. Almost every "num_predict does nothing" report is one layer quietly overriding another.

num_predict is not num_ctx

These two options are confused more than any other pair in Ollama, and the confusion costs real debugging time.

num_ctx is how much the model can read. It is the size of the context window, which holds the prompt plus everything produced so far. Raising it costs memory, because the key/value cache the model keeps for those tokens grows with the window. Sizing num_ctx for your hardware is a separate job with its own failure modes.

num_predict is how much the model will write. It is a stopping rule, not an allocation. Raising it costs wall-clock time rather than RAM, and nothing is reserved in advance.

They meet in one place. Generated tokens land inside the context window as they are produced, so a reply can also stop because the window filled rather than because your cap was reached. Ollama reports length in both cases, so the number that separates them is eval_count, covered further down.

Set it once with a Modelfile

A Modelfile bakes the value into a model you create. Write the file:

FROM qwen3:8b
PARAMETER num_ctx 8192
PARAMETER num_predict 512

Then build it and read back what you built:

ollama create qwen3-capped -f Modelfile
ollama show --parameters qwen3-capped

ollama show --parameters prints one line per stored parameter with its value. If num_predict is missing from that output, the model carries no baked-in cap and Ollama's own default applies. ollama show --modelfile qwen3-capped prints the whole definition, which is also the quickest way to copy the parameters an existing model already ships with.

This is the right layer for a value you want every caller to inherit. It is the wrong layer if you expect it to be final, because it is not.

Set it per request in the options object

Every generation endpoint takes an options object, and num_predict goes inside it:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Explain what a reverse proxy does.",
  "stream": false,
  "options": { "num_predict": 128 }
}'

/api/chat uses the same options key with the same meaning. A value here applies to that one call and to nothing else. This is the layer your tools use: a chat front end, a script, an SDK wrapper, a coding agent. They all send an options object, whether or not they show you a box for it.

Set it for one session with /set parameter

Inside ollama run, the interactive session sets options for the rest of that session:

>>> /set parameter num_predict 256
>>> /show parameters

/show parameters prints what the session will send with your next message, which makes it the fastest way to confirm a change took effect. The value lives until you type /bye. To keep it, /save qwen3-capped writes the current session, parameters included, as a new model. Nothing you /set here reaches any other client.

Which setting wins, and why yours looks ignored

The order is short. Options sent with the request beat everything else. A PARAMETER num_predict line in the model's Modelfile is the fallback used when the request carries no value. With neither, Ollama's built-in default applies.

/set parameter is not a third rule. The interactive session is an API client, so what you set there is sent as that request's options, which is exactly why it overrides the Modelfile for the session.

Now the failure this explains. You add PARAMETER num_predict 512, rebuild the model, and replies still run to thousands of tokens. Your setting is present, and ollama show --parameters proves it. It is being overridden on every request, because the client sends its own options object carrying its own number, often a number you typed into a settings screen months ago and forgot. ollama show reads the stored model. It cannot show you what arrives over HTTP.

Prove the server side in one command. Send a request that will produce a long answer, force the cap low, and read two fields:

curl -s http://localhost:11434/api/generate -d '{
  "model": "qwen3-capped",
  "prompt": "Describe the Linux boot process in detail.",
  "stream": false,
  "options": { "num_predict": 32 }
}' | jq '.done_reason, .eval_count'

That should print "length" and 32. Install jq first with sudo apt install -y jq if it is missing. A response of "length" and 32 means the server honours the option and your application is sending something different. For the server's own account of a request, restart it with OLLAMA_DEBUG=1 in the environment and watch journalctl -u ollama -f while your application talks to it.

The negative values, and the numbers you should not copy

num_predict also accepts negative values, and those are sentinels rather than counts. One negative value means "do not cap this, keep generating". Another has meant "fill the remaining context". As of August 2026 the Ollama Modelfile reference gives the default as -1, infinite generation, and earlier versions of the same table also listed -2 for filling the context.

Treat all of that as version-dependent, because it has moved. The reference documented the default as 128 for a long time before the entry was corrected at the end of 2024, so plenty of guides still repeat the old number. Read the Modelfile parameter reference for the version you actually run, then confirm the behaviour with the eval_count check above. A value you verified on your own box beats a value you read anywhere, this post included.

Why output length is the main cost on a CPU-only VPS

Generation has two phases at two very different speeds. Prompt tokens are evaluated in batches, many at a time. Output tokens are produced one at a time, and each one needs a full pass over the model weights. On a CPU-only VPS that pass is limited by memory bandwidth, so one generated token costs far more than one prompt token.

Ask for a response without streaming and the numbers are right there:

"prompt_eval_count": 26,
"prompt_eval_duration": 107345000,
"eval_count": 237,
"eval_duration": 4289432000

Durations are in nanoseconds. In that block, which is the sample response published in the Ollama API documentation rather than a measurement of any particular server, 26 prompt tokens took about 0.1 seconds while 237 output tokens took about 4.3 seconds. Your own generation rate is eval_count divided by eval_duration converted to seconds, and measuring tokens per second on your own hardware is worth doing once before you tune anything else.

The arithmetic does the rest. At 8 tokens per second, a 2,000 token answer holds the machine for more than four minutes, and the model has no idea you wanted a paragraph. Some models also loop, repeating a phrase until something stops them. With no cap, that single request keeps a core busy until the context window runs out. num_predict is the setting that bounds it, which matters most on a small self-hosted Ollama VPS where one long request is the whole machine.

Truncated output is usually the cap, not a broken model

The symptoms look like model failure. An answer that stops mid-sentence. JSON that will not parse, because the closing brace never arrived. The reflex is to blame the model or the quantisation. Read the response first.

done_reason answers the question directly. stop means the model finished on its own, either by emitting its end-of-sequence token or by matching one of the strings in your stop option. length means generation was cut off because it ran out of room. When you see length, compare eval_count with your cap: an exact match means num_predict stopped it, and a smaller number means the context window filled first.

When you stream, those fields arrive in the final chunk, the one carrying "done": true. Many client libraries discard that chunk and hand your code only the text, which is why the same truncation looks unexplained inside an application and obvious under curl. If a library is hiding it, send one request with curl to find out what the server really said.

One more point saves a wasted afternoon. Raising num_predict does not make a model write more. It only removes a ceiling. If a reply ends at 200 tokens with done_reason of stop, the model decided it was finished, and a larger cap changes nothing. Short answers with stop are a prompting problem. Short answers with length are a cap problem.

Choosing a value

• For interactive chat, leave it uncapped and press Ctrl+C to stop a runaway reply. You are watching the screen anyway.
• For anything scripted, set it. An uncapped generation inside a loop is how a batch job that should take ten minutes is still running the next morning.
• For structured output, set the cap above the largest valid document you expect, then treat done_reason of length as a hard error and retry instead of parsing what came back.
• For a coding agent, the value belongs in the agent's own configuration, because the agent sends its own options on every request. Pointing a coding agent at Ollama covers where those settings live.

The cap counts tokens, not words and not characters, so do not estimate it. Generate one representative answer with no cap, read eval_count, and set the limit comfortably above that. Model families tokenise differently, so a value that fits a Llama model may truncate the same answer from a Qwen 3 model on the same VPS.

FAQ

What is the difference between num_ctx and num_predict in Ollama?

num_ctx is the size of the context window, so it sets how much the model can read: the prompt plus everything produced so far. It costs memory, because the key/value cache grows with it. num_predict sets how many tokens the model may write in one response. It costs time rather than memory, and nothing is reserved in advance. Generated tokens count against both, so a reply can be cut short by either one.

Why does my num_predict setting seem to be ignored?

Because a value sent with the request overrides a value stored in the model. Put PARAMETER num_predict 512 in a Modelfile, then drive that model from a chat front end or a coding agent, and the client sends its own options object whose number wins. ollama show --parameters still prints your value, because it reads the stored model and cannot see what arrives over HTTP. Send one request with curl using "options": {"num_predict": 32} and check that eval_count comes back as 32, which confirms the server itself is behaving and moves the search into your application.

How can I tell whether my output was cut off by num_predict?

Send the request with "stream": false and read done_reason. A value of stop means the model finished on its own. A value of length means it ran out of room. Then compare eval_count with your cap: if they match exactly, num_predict stopped it, and if eval_count is smaller, the context window filled first. When streaming, both fields arrive in the final chunk with "done": true, which many client libraries throw away before your code sees it.

What is the default value of num_predict?

Read it from your own install rather than from an article. As of August 2026 the Ollama Modelfile reference gives the default as -1, meaning generation is not capped, and that entry was corrected at the end of 2024 after years of documenting 128. Negative values are sentinels rather than counts, and older versions of the same table also listed -2 for filling the remaining context. Check the Modelfile parameter reference for your version, then confirm it with ollama show --parameters and one curl request.

Does raising num_predict make the model write longer answers?

No. It only removes a ceiling. If a reply ends with done_reason of stop, the model decided it was finished, and a larger cap changes nothing. Length in that case is a prompting question: ask for a specific structure, a section count, or a stated level of detail. Raise num_predict only when done_reason comes back as length.