Ollama num_predict: How to Limit Output Tokens

Wetin num_predict dey do for Ollama

num_predict na Ollama option wey dey limit how many tokens model fit generate for one response. E dey count output tokens only, so prompt no dey count for am. When model reach the limit, generation go stop for where e dey, sometimes for middle of word, and response go return with done_reason set to length.

Na the complete feature be that. The wahala be say Ollama give you three different places wey you fit set the value, and the setting wey dey nearest to request na im go win. Almost every report say "num_predict no dey do anything" happen because one layer quietly override another.

num_predict no be num_ctx

People dey mix up these two options pass any other pair for Ollama, and this confusion dey waste real debugging time.

num_ctx na how much the model fit read. Na the size of the context window, wey dey hold the prompt plus everything wey e don produce so far. If you raise am, e go use more memory, because the key/value cache wey the model dey keep for those tokens go grow as the window grow. How to size num_ctx for your hardware na separate work with im own failure modes.

num_predict na how much the model go write. Na stopping rule, no be allocation. If you raise am, e go cost more wall-clock time instead of RAM, and nothing dey reserved ahead of time.

Dem meet for one place. As the model dey produce tokens, the generated tokens enter the context window. So reply fit stop because the window don full, instead of because e reach your cap. Ollama reports length for both cases, so eval_count na the number wey separate dem. We cover am further down.

Set am once with a Modelfile

A Modelfile dey bake the value inside model wey you create. Write the file:

FROM qwen3:8b
PARAMETER num_ctx 8192
PARAMETER num_predict 512

Then build am and read wetin you build:

ollama create qwen3-capped -f Modelfile
ollama show --parameters qwen3-capped

ollama show --parameters dey print one line for each stored parameter and e value. If num_predict no dey that output, the model no get any baked-in cap and Ollama own default go apply. ollama show --modelfile qwen3-capped dey print the complete definition. This one na the fastest way to copy the parameters wey existing model already ship with.

This na the correct layer for value wey you want every caller to inherit. E no be the correct layer if you expect the value to be final, because e no be final.

Set am for each request inside the options object

Every generation endpoint dey take an options object, and num_predict dey inside am:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Explain what a reverse proxy does.",
  "stream": false,
  "options": { "num_predict": 128 }
}'

/api/chat dey use the same options key with the same meaning. Any value wey you set here go apply to only that call, and nothing else. Na this layer your tools dey use: a chat front end, a script, an SDK wrapper, or a coding agent. All of dem dey send an options object, whether dem show you a box for am or not.

Set am for one session with /set parameter

Inside ollama run, the interactive session dey set options for the rest of that session:

>>> /set parameter num_predict 256
>>> /show parameters

/show parameters go print wetin the session go send with your next message. This na the fastest way to confirm say the change don take effect. The value go remain until you type /bye. To keep am, /save qwen3-capped go write the current session, including the parameters, as a new model. Nothing wey you /set here go reach any other client.

Which setting dey win, and why e be like say your own no dey work

The order short. Options wey request send go override everything else. A PARAMETER num_predict line for the model Modelfile na fallback wey e go use when request no carry value. If both no dey, Ollama built-in default go apply.

/set parameter no be third rule. Interactive session na API client, so wetin you set there dey go as that request options. Na exactly why e dey override Modelfile for the session.

Now, na failure wey this one explain. You add PARAMETER num_predict 512, rebuild the model, but replies still dey run reach thousands of tokens. Your setting dey present, and ollama show --parameters prove am. E dey get overridden for every request because client dey send im own options object with im own number. Many times, na number wey you type for settings screen months ago and forget. ollama show dey read the stored model. E no fit show wetin arrive through HTTP.

Prove the server side with one command. Send request wey go produce long answer, force the cap make e low, then read two fields:

curl -s http://localhost:11434/api/generate -d '{
  "model": "qwen3-capped",
  "prompt": "Describe the Linux boot process in detail.",
  "stream": false,
  "options": { "num_predict": 32 }
}' | jq '.done_reason, .eval_count'

That one suppose print "length" and 32. Install jq first with sudo apt install -y jq if e no dey. Response of "length" and 32 mean say server dey honour the option, and your application dey send different thing. To see the server own record of request, restart am with OLLAMA_DEBUG=1 for the environment, then monitor journalctl -u ollama -f while your application dey talk to am.

Negative values, and numbers wey you no suppose copy

num_predict still accepts negative values, and dem be sentinels, no be counts. One negative value mean “no cap for this; continue generating”. Another one don mean “fill the remaining context”. As of August 2026, Ollama Modelfile reference give -1 as the default for infinite generation. Older versions of the same table also list -2 for filling the context.

Treat all these values as version-dependent because dem don change. The reference document the default as 128 for a long time before dem correct the entry at the end of 2024. Because of that, plenty guides still repeat the old number. Read the Modelfile parameter reference for the version wey you dey run, then confirm the behaviour with the eval_count check above. Value wey you verify for your own box better pass any value wey you read anywhere, including this post.

Wetin make output length be the main cost for CPU-only VPS

Generation get two phases wey dey run for very different speeds. Prompt tokens dey process in batches, plenty at once. Output tokens dey produce one by one, and each one need full pass through the model weights. For CPU-only VPS, memory bandwidth dey limit this pass, so one generated token cost far more than one prompt token.

Ask for response without streaming, and you go see the numbers directly:

"prompt_eval_count": 26,
"prompt_eval_duration": 107345000,
"eval_count": 237,
"eval_duration": 4289432000

Durations dey for nanoseconds. For that block, wey na sample response published for Ollama API documentation, not measurement from any particular server, 26 prompt tokens take about 0.1 seconds, while 237 output tokens take about 4.3 seconds. Your own generation rate na eval_count divided by eval_duration, then convert am to seconds. E good make you measure tokens per second for your own hardware once before you tune anything else. That rate depend on the model as much as e depend on the machine. So if long answers na the real cost, model wey dem build for fast decoding, like Nemotron 3.5 Lightning for VPS, fit recover some of the time wey low cap dey otherwise protect.

The arithmetic go show the rest. For 8 tokens per second, 2,000-token answer go keep the machine busy for more than four minutes, and the model no know say you want only one paragraph. Some models fit also loop, repeating one phrase until something stop dem. Without cap, that one request go keep one core busy until the context window finish. num_predict na the setting wey limit am. This matter pass for small self-hosted Ollama VPS, where one long request fit occupy the whole machine.

Output wey cut short na usually cap, no be model don spoil

The signs fit look like model failure. Answer wey stop halfway through sentence. JSON wey no go parse because closing brace no reach. First reaction na to blame the model or the quantisation. Read the response first.

done_reason mean say answer don respond the question directly. stop mean say model finish by itself, either e emit end-of-sequence token or e match one of the strings for your stop option. length mean say generation stop because space don finish. When you see length, compare eval_count with your cap: if dem match exactly, num_predict stop am, but if number small pass, context window fill first.

When you stream, those fields go arrive for final chunk, the one wey carry "done": true. Plenty client libraries dey discard that chunk and give your code only the text. Na why the same truncation go look unexplained inside application, but e go clear under curl. If library dey hide am, send one request with curl to know wetin server really talk.

One more thing fit save you from wasting afternoon. Increasing num_predict no make model write more. E only remove one ceiling. If reply end for 200 tokens with done_reason of stop, model decide say e don finish, and bigger cap no go change anything. Short answers with stop na prompting problem. Short answers with length na cap problem.

Value wey you go choose

• For interactive chat, leave am uncapped and press Ctrl+C to stop reply wey no dey end. You dey watch the screen anyway.
• For anything scripted, set am. Uncapped generation inside loop na how batch job wey suppose take ten minutes still dey run the next morning.
• For structured output, set the cap above the biggest valid document wey you expect. Then treat done_reason of length as hard error and retry instead of parsing wetin come back.
• For coding agent, the value belong inside the agent own configuration, because the agent dey send im own options for every request. Pointing coding agent go Ollama cover where those settings dey.

The cap dey count tokens, no be words or characters, so no estimate am. Generate one representative answer with no cap, read eval_count, then set the limit comfortably above that. Model families dey tokenise differently, so value wey fit Llama model fit truncate the same answer from Qwen 3 model for the same VPS.

FAQ

Wetin be di difference between num_ctx and num_predict for Ollama?

num_ctx na di size of di context window, so e set how much di model fit read: di prompt plus everything wey e don produce so far. E dey use memory, because di key/value cache dey grow with am. num_predict set how many tokens di model fit write for one response. E dey use time instead of memory, and nothing dey reserve ahead of time. Generated tokens count for both, so either one fit cut reply short.

Why my num_predict setting dey look like say e no work?

Na because value wey request send go override value wey dey stored for di model. Put PARAMETER num_predict 512 for Modelfile, then use chat front end or coding agent take run dat model, and di client go send im own options object. Di number wey client send na im go win. ollama show --parameters still print your value, because e dey read di stored model and e no fit see wetin arrive through HTTP. Send one request with curl using "options": {"num_predict": 32} and check say eval_count come back as 32. Dis confirm say di server itself dey behave well, and move di search go your application.

How I fit know whether num_predict cut my output short?

Send di request with "stream": false and read done_reason. Value of stop mean say di model finish by itself. Value of length mean say e run out of space. Then compare eval_count with your cap: if dem match exactly, num_predict stop am. If eval_count small pass, di context window fill first. When streaming, both fields dey arrive for di final chunk with "done": true, but plenty client libraries dey throw am away before your code fit see am.

Wetin be di default value of num_predict?

Read am from your own install instead of article. As of August 2026, Ollama Modelfile reference give di default as -1. Dis mean say generation no get cap. Dem correct dat entry for end of 2024 after dem don document 128 for years. Negative values na sentinels, no be counts. Older versions of di same table also list -2 for filling di remaining context. Check di Modelfile parameter reference for your version, then confirm am with ollama show --parameters and one curl request.

If I raise num_predict, e go make di model write longer answers?

No. E only remove ceiling. If reply end with done_reason of stop, di model decide say e don finish, and bigger cap no go change anything. For dis case, length na prompting matter: ask for specific structure, section count, or stated level of detail. Raise num_predict only when done_reason come back as length.