How Reasoning Effort Dey Work for Local LLM

Reasoning effort dey change wetin for local LLM

Reasoning effort na setting wey tell model how long e go think before e answer. E dey change the length of the reasoning segment only; e no change anything else. The weights wey dey disk na the same for every level, the quantisation na the same, and answer still dey come from the same forward pass. The thing wey change na how many tokens model go first spend for im own scratchpad.

This difference matter because of where those tokens go. For hosted API, reasoning tokens go show for invoice. For VPS wey you own, na generation time for your own CPU or GPU you dey pay with, plus space inside the context window. If you leave model for the highest effort, e fit spend most of im output on reasoning before the first word of answer show. For self-hosted hardware, this fit be the difference between reply wey take two seconds and one wey take two minutes.

Level dey where: chat template, no be weights

Thinking model dey trained to first produce reasoning segment, usually dem dey wrap am with <think> and </think> tags, before e give final answer. Effort level na instruction wey model chat template dey write inside prompt. That template na Jinja file wey come with the model. E dey read variable like reasoning_effort and render different system-level line for each value. Model train to make im scratchpad short or long based on that line.

Two things follow from this. Level names belong to the model, no be your runtime. So name from one model card fit mean nothing for another model. And if anything for the chain replace model chat template with generic one, variable no go render and setting go silently do nothing.

For 2026-10-12, Qwen3.8-27B model card document three effort levels: low, medium and xhigh, with xhigh as default. No high dey. You switch thinking with enable_thinking, and e dey on by default. The card also document preserve_thinking, wey dey on by default, to keep reasoning from earlier turns inside conversation history. gpt-oss dey use low, medium and high instead. Plenty other model families accept boolean only, with nothing else. Read the card for the exact version wey you pull, because these names no be standard. How to run 27B model for VPS come first. This page na about wetin to set after e don answer.

Why high effort dey cost more for VPS

Output tokens. Reasoning tokens na generated tokens. Dem dey pass through the same decode loop like answer, for the same tokens-per-second speed wey your hardware fit manage. Suppose task produce 200 answer tokens and 4,000 reasoning tokens. You generate 4,200 tokens, but reader see only 200. Your decode rate depend on memory bandwidth, the quantisation wey you pick and your thread count: llama.cpp own tuning notes warn say --threads value wey pass your physical core count fit oversaturate CPU and make generation slow well-well. Once you tune these things, the only lever wey remain na the token count itself.

Wall clock. Person dey wait for the first token of the answer, because everything before that na blank screen or collapsed spinner. Reasoning dey come out first, so the wait roughly na reasoning token count divided by your decode rate, plus prompt processing. If you double the reasoning length, you double that wait.

Context. Reasoning tokens dey occupy context window like any other token. When preserve_thinking dey on, the scratchpad from turn one still dey inside prompt for turn five, so prompt processing dey slow down every turn while the window dey fill from both ends. If you raise num_ctx to hold am e go cost KV cache memory, and for VPS wey no get GPU, na system RAM wey you fit no get extra.

Wen to raise am, and wen to leave am low

Raise am for work wey wrong step for middle fit spoil the final result: arithmetic and unit conversion wey get many steps, planning edit across different files, code wey must compile, and constraint problems wey one answer must satisfy many conditions at once. For these ones, the scratchpad dey do real work, and longer scratchpad na cheap way to catch error wey model fit otherwise commit to.

Leave am low wen answer don dey inside the input and na to move am be the work. Extraction, classification, tagging, translation, rewriting, summarising and formatting all dey this group. The reasoning part mostly dey repeat the task, and e dey give model space to argue itself comot from correct first instinct.

Leave am low for anything interactive too. For chat box or editor, you dey inside the loop, so fast answer wey you fit correct better pass slow one wey you must wait for. Na this be the real trade-off behind pointing coding agent at local model: agent dey make plenty small calls, and dem dey charge the reasoning tax for every one of dem.

How to set the level in llama.cpp

llama.cpp dey write the variable directly inside template, so na the runtime wey you fit confirm say the level reach. Point -m to the GGUF wey you already get.

llama-server -m ./qwen3.8-27b-Q4_K_M.gguf \
  --jinja \
  --reasoning-effort medium \
  --reasoning-format deepseek \
  -c 32768 \
  --host 127.0.0.1 --port 8080

--jinja dey use the model own chat template, and current builds enable am by default. --reasoning-effort fit accept default, minimal, low, medium, high, xhigh or max. default mean say make e leave the template own default as e be. This list na llama.cpp vocabulary, no be the model own vocabulary, so pass only name wey the card list: if the template no define the level, request fit fail with template error. --reasoning-format deepseek move the reasoning from message.content enter message.reasoning_content. Na this one make the split measurable for the next section.

If you wan switch thinking off instead of making am shorter, set the template variable by yourself:

llama-server -m ./qwen3.8-27b-Q4_K_M.gguf --jinja \
  --chat-template-kwargs '{"enable_thinking": false}'

--reasoning-budget na different mechanism. E cap the reasoning segment by token count. 0 stop am immediately, while -1 leave am without limit. E no dey ask the model to plan a shorter segment. Both flags set the server default. As of 2026-10-12, llama-server /v1/chat/completions also accept reasoning_effort and chat_template_kwargs inside request body. E pass the level to template for that one call, while none switch reasoning off. So one process fit serve two effort levels.

vLLM expose the same variable for each request, inside the OpenAI-compatible body:

{"model": "Qwen/Qwen3.8-27B",
 "messages": [{"role": "user", "content": "Summarise this changelog in two lines."}],
 "chat_template_kwargs": {"reasoning_effort": "medium"}}

Ollama for level ken set

Ollama get im own field, think, for /api/chat and /api/generate. E dey accept true, false, or level name. As of 2026-10-12, for model wey dey publish thinking metadata, na the model own names dem be: /api/show dey list dem under thinking.values, with the default beside dem. If model no list one name, e go fall back to that default without error. Models wey no get the metadata dey use Ollama old fixed set of low, medium, high and max. If you leave think out, the model own default go apply. For Qwen3.8, na thinking on.

ollama run qwen3.8:27b --think=low "Draft a one line commit message for a README typo fix"

{"model": "qwen3.8:27b",
 "messages": [{"role": "user", "content": "Which HTTP status code means the request body was too large?"}],
 "think": "low",
 "stream": false}

The reasoning go come back for message.thinking, and the answer for message.content. Ollama don already split dem for you. Inside interactive ollama run session, /set think and /set nothink fit toggle am without restart, while /set think low go choose level.

Now notice where this fit fail quietly. Qwen3.8 template define low, medium and xhigh. Ollama model carry template wey dem package inside its tag, instead of the Jinja file from the original repository. Ollama go pass level only if that packaged model list am. First check wetin your tag advertise:

curl -s http://localhost:11434/api/show -d '{"model": "qwen3.8:27b"}' | jq .thinking

If name no dey inside values, like high or max for this model, e go run with the default instead. No assume say e work. To measure am, na about one minute.

How to measure whether the level actually landed

Send the same prompt at more than one level with temperature at 0, then compare the token counts. The three names below na the ones Qwen3.8 card list; if /api/show list different ones for your tag, use those ones. Here jq dey build the body so you no need escape quotes by hand.

for level in low medium xhigh; do
  body=$(jq -n --arg lvl "$level" '{
    model: "qwen3.8:27b",
    messages: [{role: "user", content: "A pump fills a 4500 litre tank in 25 minutes. A second pump is 40 percent slower. How long do both together take? Answer in minutes."}],
    think: $lvl,
    stream: false,
    options: {temperature: 0, num_ctx: 8192}
  }')
  echo "== $level"
  curl -s http://localhost:11434/api/chat -d "$body" | jq '{
    thinking_chars: (.message.thinking // "" | length),
    answer_chars: (.message.content | length),
    eval_count: .eval_count,
    seconds: (.total_duration / 1e9),
    tok_per_sec: (.eval_count / (.eval_duration / 1e9))
  }'
done

eval_count na every token wey dem generate, reasoning join, so the gap between two levels na almost all reasoning. thinking_chars dey give you the split directly. Two things suppose happen: the numbers go change between levels, and the answer go still correct for the lower one. If eval_count remain within noise for all three runs, the level dey ignored. Check the names against /api/show first, because Ollama dey run name wey no dey list with the default; if dem match, the fix na runtime wey dey pass the level.

Total time na only half the matter, so measure the gap reach the first answer token by streaming and stopping for the first content chunk wey no empty. This one need jq and bc.

start=$(date +%s.%N)
curl -sN http://localhost:11434/api/chat -d '{
  "model": "qwen3.8:27b",
  "messages": [{"role": "user", "content": "Explain what a reverse proxy does, in three sentences."}],
  "think": "low",
  "stream": true
}' |
while IFS= read -r line; do
  if [ -n "$(printf '%s' "$line" | jq -r '.message.content // ""')" ]; then
    echo "first answer token after $(echo "$(date +%s.%N) - $start" | bc)s"
    break
  fi
done

Run am for low and run am again for xhigh. The difference na the wait wey you dey buy. For llama.cpp, the same numbers dey come back inside the response, and you no need shell arithmetic:

curl -s http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "local", "temperature": 0,
       "messages": [{"role": "user", "content": "A pump fills a 4500 litre tank in 25 minutes. A second pump is 40 percent slower. How long do both together take?"}]}' | jq '{
  reasoning_chars: (.choices[0].message.reasoning_content // "" | length),
  answer_chars: (.choices[0].message.content | length),
  predicted_n: .timings.predicted_n,
  tok_per_sec: .timings.predicted_per_second
}'

Do this for your own box. Published effort comparison na hardware wey no be your own, and your decode rate na the term wey turn token count to seconds. Measure tokens per second for your own server go give you that term: reasoning tokens divided by your decode rate na the wait wey you just add.

Wetin fit go wrong

The answer cut off, or content empty while thinking full. The generation limit don finish because of reasoning. Ollama num_predict dey cap the whole generation, including reasoning, and reasoning dey happen first. So, cap of 512 tokens for high effort fit end the response before the answer start. Ollama dey report "done_reason": "length" for that response. Raise the cap or reduce the effort. How num_predict dey count tokens explain the interaction well.

The level no change anything. Token counts dey the same for every level. Either the runtime no dey pass the variable, the template no dey read am, or for Ollama, the name no dey among the names wey the tag list, so e fall back to the default. Check the template wey your runtime dey really use, instead of the one for the original repository. llama.cpp with --jinja and --chat-template-kwargs dey write the variable by hand, so e good for control test. If the level work there but no work anywhere else, the model dey fine and the other runtime dey drop am.

The system reject level name. If template error happen when you send request, or the first message fail while the server dey otherwise healthy, e usually mean say you pass level wey the template no define. Example na high for model wey the card list only low, medium and xhigh. For Ollama, model wey no get thinking metadata dey accept only low, medium, high and max. Anything else go return invalid think value error.

Multi-turn chats dey slow down for every turn. The history still dey keep old reasoning. Set preserve_thinking to false if the model support am, or remove thinking field from the messages wey you send back. If you no do this, prompt processing go grow for every turn while the answers remain the same length.

Quality dey drop for low effort on task wey you think say simple. Some extraction no be just extraction. If the input need unit conversion or rule wey you must apply in order, na reasoning task with short output. Raise the level for that one call, instead of raising am for the whole server.

Run levels two at once

llama.cpp dey set default level when e start with --reasoning-effort, but current builds fit also take reasoning_effort for each request. So one process and one copy of the weights fit serve one editor with low level and one nightly batch job with higher level. If you prefer set each level when dem start, two processes for two ports go work too. But this one need two copies of the weights for memory unless you run the jobs at different times. Any way, both jobs dey share one machine decode capacity. For one VPS, the cheaper setup normally na low effort for anything wey person dey wait for, plus one scheduled higher effort run for work wey nobody dey monitor. Wetín dey happen when several users share one local model apply here too: reasoning tokens na decode work. So if you increase the effort, your effective concurrency go reduce by roughly the same factor wey e increase the token count.

FAQ

Which reasoning effort level should I use by default?

For default, start with the lowest level wey the model get, and only raise am for tasks wey you don watch fail. Several thinking models ship with high default, and Qwen3.8-27B defaults to xhigh, wey be the highest level, as of October 2026. Dem choose that default to look good for benchmark tables, but benchmark table no charge for time. For your own hardware, na seconds you dey pay, so make higher level be something wey you choose per task, instead of setting wey every request inherit.

Do reasoning tokens count against my context window?

Yes. Dem be ordinary tokens for the output, and dem dey inside context window together with everything else. Whether dem remain there for the next turn depend on the runtime and the model. Qwen3.8 card document preserve_thinking, and e dey on by default. This one keeps earlier reasoning inside the history, so long conversation go carry every scratchpad wey e don produce. Set am to false, or remove thinking field from the messages wey you replay, and prompt processing no go continue grow.

Why does changing the thinking level make no difference to my token counts?

The setting no dey reach chat template. The level na template variable, so e go work only if runtime pass am and the packaged template read am. Some runtimes ship their own template together with model instead of the Jinja file from the original repository. For this reason, the variable fit disappear without any error showing anywhere. Ollama too dey run a level name wey the model no list at the model default, again without error. Prove am by sending the same prompt at the lowest and highest level, with temperature set to 0, then compare eval_count. If the counts match within noise, the level dey ignored.

Does lower reasoning effort make the model less accurate?

E depend on the task, and you suppose measure am instead of assuming. Where the answer already dey inside the input, like extraction or rewriting, shorter scratchpad usually no change anything. Where an intermediate step must correct before the final one fit correct, like multi-step arithmetic or code wey must compile, accuracy dey drop when scratchpad shorter. Build set of twenty prompts from your real workload, run dem at two levels with temperature set to 0, then count the wrong answers. That number specific to your workload, and no published table fit give am to you.