SSD Nodes Learn Hosting plans →
How to do am Matt ConnorBy Matt Connor · Updated 2026-10-26

Ollama num_predict: How to Limit Token Output

Learn how Ollama num_predict limits output tokens, which of the three settings wins, and how done_reason shows when generation stops at the cap.

num_predict dey do wetin for Ollama

num_predict na Ollama option wey dey limit how many tokens model fit generate for one response. E dey count output tokens only, so prompt no dey count inside am. When model reach the limit, generation go stop for where e dey, sometimes for middle of word, and response go come back with done_reason set to length.

Na everything this feature dey do. The hard part be say Ollama give you three different places to set the value, and setting wey dey closest to the request go win. Almost every report say “num_predict no dey do anything” happen because one layer dey quietly override another one.

num_predict no be num_ctx

People dey confuse these two options pass any other pair for Ollama, and this confusion dey waste real debugging time.

num_ctx na how much the model fit read. Na the size of the context window, wey hold the prompt plus everything wey e don produce so far. If you raise am, e go use more memory because the key/value cache wey the model dey keep for those tokens dey grow with the window. How to size num_ctx for your hardware na separate work with its own ways to fail.

num_predict na how much the model go write. Na stopping rule, no be allocation. If you raise am, e go cost more wall-clock time instead of RAM, and nothing dey reserve ahead of time.

Dem meet for one place. As the model dey produce tokens, dem enter inside the context window. For Ollama v0.40.11, context shift dey on by default for most models, so when the window full, older tokens dey drop and generation continue. If shift dey off because the request send "shift": false or the model no support shifting, reply fit still stop because the window full instead of because e reach your cap. Ollama dey report length for both cases, so eval_count na the number wey dey separate dem. We go cover am further down.

Set am one time with a Modelfile

A Modelfile dey bake the value inside the model wey you create. Write the file:

FROM qwen3:8b
PARAMETER num_ctx 8192
PARAMETER num_predict 512

Then build am and read back wetin you build:

ollama create qwen3-capped -f Modelfile
ollama show --parameters qwen3-capped

ollama show --parameters dey print one line for each stored parameter together with the value. If num_predict no dey for that output, the model no get baked-in cap, and Ollama own default go apply. ollama show --modelfile qwen3-capped dey print the complete definition. This one na also the fastest way to copy the parameters wey existing model already ship with. Creating capped model this way dey use almost no extra disk space, because the new entry dey reuse the weight blobs wey the base model don already download instead of copying dem. You suppose know where Ollama dey keep those blobs before VPS root disk go fill up.

This na the correct layer for value wey you want every caller to inherit. But e no be the correct layer if you expect am to be final, because e no be final.

Set am for each request inside the options object

Every generation endpoint dey take an options object, and num_predict dey inside am:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Explain what a reverse proxy does.",
  "stream": false,
  "options": { "num_predict": 128 }
}'

/api/chat dey use the same options key with the same meaning. Any value wey you put here go apply only to that call and nothing else. Na this layer your tools dey use: chat front end, script, SDK wrapper, coding agent. All of dem dey send an options object, whether dem show you a box for am or not.

Set am for one session with /set parameter

Inside ollama run, the interactive session dey set options for the remaining part of that session:

>>> /set parameter num_predict 256
>>> /show parameters

/show parameters go print wetin the session go send with your next message. This one na the fastest way to confirm say the change don take effect. The value go remain until you type /bye. To keep am, /save qwen3-capped go write the current session, including the parameters, as new model. Nothing wey you /set here go reach any other client.

Which setting dey win, and why e look like say yours no work

The order short. Options wey request send dey beat everything else. A PARAMETER num_predict line for the model Modelfile na fallback wey dem use when request no carry value. If both no dey, Ollama built-in default go apply.

/set parameter no be third rule. The interactive session na API client, so wetin you set there dey go as that request options. Na exactly why e dey override Modelfile for the session.

Now na the failure wey this one explain. You add PARAMETER num_predict 512, rebuild the model, but replies still dey run reach thousands of tokens. Your setting dey there, and ollama show --parameters prove am. Every request dey override am, because client dey send im own options object with im own number. Plenty times, na number you type for settings screen months ago and forget. ollama show dey read the stored model. E no fit show wetin arrive through HTTP.

Prove the server side with one command. Send request wey go produce long answer, force the cap make e low, then read two fields:

curl -s http://localhost:11434/api/generate -d '{
  "model": "qwen3-capped",
  "prompt": "Describe the Linux boot process in detail.",
  "stream": false,
  "options": { "num_predict": 32 }
}' | jq '.done_reason, .eval_count'

Detailed answer go pass 32 tokens by far, so expect "length" and 32. Install jq first with sudo apt install -y jq if e no dey. The 32 no guaranteed, because eval_count na only the number of output tokens wey server actually generate. If model finish early, you go get "stop" and smaller count. The fix na prompt wey ask for more. When you get "length" and 32, the request own value beat the 512 wey dey stored for qwen3-capped. This show say server dey honour per-request options. E no tell you wetin your application dey send, so now search move go client. To see the server own account of request, restart am with OLLAMA_DEBUG=1 for the environment, then monitor journalctl -u ollama -f while your application dey talk to am.

The negative values, and the numbers you should not copy

num_predict still dey accept negative values, and dem be sentinels instead of counts. One negative value mean “no cap for this, keep generating”. Another one don mean “fill the remaining context”. As of October 2026, the Ollama Modelfile reference dey give the default as -1, meaning infinite generation. Older versions of the same table also list -2 for filling the context.

Treat all these values as version-dependent, because dem don change. For long time, the reference document the default as 128 before dem correct the entry to -1. Na why plenty guides still dey repeat the old number. The behaviour don change too. Ollama v0.40.11, wey be the current release on October 25, 2026, dey pass GGUF models to an upstream llama-server process wey need finite budget. Because of this, e replace any negative value, and any cap above ten times num_ctx, with ten times num_ctx. So, “infinite” mean 81,920 tokens for the qwen3-capped model above, and -2 no longer behave differently from -1. Read the Modelfile parameter reference for the version wey you dey run, then confirm the behaviour with the eval_count check above. Value wey you verify for your own box better pass value wey you read anywhere, including this post.

Why output length be the main cost for CPU-only VPS

Ollama free to run for machine wey you control, so local generation no dey get usage meter, and long answer na time you go pay for instead. Generation get two phases wey speed dey very different. Prompt tokens dey process in batches, many at once. Output tokens dey produce one by one, and each one need one pass through the weights wey e dey use. For CPU-only VPS, memory bandwidth dey limit that pass, so one generated token cost much more than one prompt token. Because that pass need read every weight wey the token dey use (all of dem for dense model, but na only the routed experts for mixture-of-experts model like Nemotron 3.5 Lightning, wey dey activate about 3B of its 30B parameters for each token), the number of bytes wey each weight occupy set the highest token rate wey you fit get. Na why q4 build dey decode faster than the same model for q8 or fp16.

Ask for response without streaming and you go see the numbers directly:

"prompt_eval_count": 11,
"prompt_eval_duration": 13074791,
"eval_count": 18,
"eval_duration": 52479709

Durations dey for nanoseconds. For that block, wey be the sample response wey dem publish for Ollama API documentation as of October 2026, no be measurement from any particular server, 11 prompt tokens take about 0.013 seconds while 18 output tokens take about 0.052 seconds. That na roughly 1.2 milliseconds per prompt token against 2.9 milliseconds per output token. So even for one-line reply, each generated token cost more than twice as much as each token wey system read. For long answer, na the output side dey use most of the time. Your own generation rate na eval_count divided by eval_duration and converted to seconds. Measure tokens per second for your own hardware once before you tune anything else. That rate depend on the model as much as e depend on the machine. So if long answers na the real cost, model wey dem build for fast decoding, like Nemotron 3.5 Lightning for VPS, fit recover some of the time wey low cap dey otherwise protect.

The arithmetic go show the rest. At 8 tokens per second, 2,000-token answer go keep the machine busy for more than four minutes, and the model no know say na paragraph you want. Reasoning model dey use part of that budget to think before e write the word wey you ask for. That thinking dey generate one token at a time like everything else. So the reasoning effort wey you request na another control for the same bill. Some models dey also loop, repeating phrase until something stop dem. Without cap, that one request go keep one core busy until e reach ten times num_ctx tokens, because context shift stop a full window from ending am. num_predict na the setting wey limit am. This matter most for small self-hosted Ollama VPS where one long request na the whole machine.

Output wey truncate usually na limit, no be broken model

The symptoms fit look like model failure. Answer wey stop halfway through sentence. JSON wey no go parse because closing brace never show. The first reaction na to blame the model or the quantisation. Read the response first.

done_reason mean say answer address the question directly. stop mean say generation end before e reach any limit, either because model emit im end-of-sequence token or because output match one of the strings for your stop option, wey go end the reply for wherever the match happen. length mean say generation stop because available space finish. When you see length, compare eval_count with your cap: exact match mean say num_predict stop am, while smaller number mean say context window fill first, and this fit only happen when context shift dey off.

When you stream, those fields go arrive for the final chunk, the one wey carry "done": true. Plenty client libraries discard that chunk and give your code only the text. Na why the same truncation dey look unexplained inside application but obvious under curl. If library dey hide am, send one request with curl to know wetin server really talk.

One more point fit save you from wasting afternoon. Increasing num_predict no dey make model write more. E only remove one ceiling. If reply end for 200 tokens with done_reason of stop, either model decide say e don finish or one of your stop strings match, and bigger cap no go change anything. Short answers with stop na prompting or stop-string problem. Short answers with length na cap problem.

Wetin value to choose

  • For interactive chat, leave am uncapped and press Ctrl+C to stop reply wey no dey end. You dey watch the screen anyway.
  • For anything scripted, set am. If generation wey no get cap dey inside loop, batch job wey suppose take ten minutes fit still dey run the next morning.
  • For structured output, set the cap pass the biggest valid document wey you expect. Then treat done_reason of length as hard error and retry, instead of parsing wetin come back.
  • For coding agent, the value belong for the agent own configuration, because the agent dey send im own options for every request. How to point coding agent to Ollama explain where those settings dey.

The cap dey count tokens, no be words and no be characters, so no estimate am. Generate one representative answer without cap, read eval_count, then set the limit comfortably above that. Model families dey tokenise differently, so value wey fit Llama model fit truncate the same answer from Qwen 3 model for the same VPS.

FAQ

Wetin be difference between num_ctx and num_predict for Ollama?

num_ctx na size of the context window, so e set how much the model fit read: the prompt plus everything wey e don produce so far. E dey use memory, because the key/value cache dey grow with am. num_predict set how many tokens the model fit write for one response. E dey use time instead of memory, and nothing dey reserve ahead. Generated tokens count for both, so if context shift off, either one fit cut the reply short.

Why my num_predict setting dey look like say e no dey work?

Because value wey request send go override value wey dey stored for the model. Put PARAMETER num_predict 512 inside Modelfile, then use that model from chat front end or coding agent, and the client go send im own options object. Na that number go win. ollama show --parameters still print your value, because e dey read the stored model and e no fit see wetin arrive through HTTP. Send one request with curl using "options": {"num_predict": 32} and prompt wey need long answer, then read done_reason and eval_count. A length with an eval_count of 32 show say the request value override the stored one. This confirm say the server dey behave well, so continue the search inside your application. A stop with smaller count only mean say the model finish first, so retry with prompt wey ask for more.

How I fit know whether num_predict cut my output short?

Send the request with "stream": false and read done_reason. Value of stop mean say the model finish by itself or one of your stop strings end am. Value of length mean say e run out of space. Then compare eval_count with your cap: if dem match exactly, num_predict stop am; if eval_count small pass, the context window fill first. When streaming, both fields dey arrive for the final chunk with "done": true, but many client libraries dey throw am away before your code see am.

Wetin be the default value of num_predict?

Read am from your own install instead of relying on article. As of October 2026, the Ollama Modelfile reference give the default as -1. This mean say generation no get cap, and dem correct that entry after dem don document 128 for long time. For Ollama v0.40.11, that open-ended default, like any negative value, actually stop at ten times num_ctx. Negative values na sentinels, no be counts, and older versions of the same table also list -2 for filling the remaining context. Check the Modelfile parameter reference for your version, then confirm am with ollama show --parameters and one curl request.

If I raise num_predict, the model go write longer answers?

No. E only remove the ceiling. If reply end with done_reason of stop, the model decide say e finish or one of your stop strings match, and bigger cap no go change anything. For that case, length na prompting matter: ask for specific structure, number of sections, or clear level of detail. Raise num_predict only when done_reason come back as length.

Dis article help you?