Wetin reasoning effort dey change for local LLM

Wetin reasoning effort dey change for local LLM

Reasoning effort na setting wey tell model how long e suppose think before e answer. E dey change the length of the reasoning segment only; e no change anything else. The weights wey dey disk na the same for every level, the quantisation na still the same, and answer still come from the same forward pass. Wetin change na how many tokens model go first spend for im own scratchpad.

This difference matter because of where those tokens dey go. For hosted API, reasoning tokens dey show for invoice. For VPS wey you own, you dey pay for dem with generation time for your own CPU or GPU, plus space inside the context window. If you leave model for the highest effort, e fit spend most of im output for reasoning before the first word of answer show. For self-hosted hardware, this fit mean say reply go take two seconds instead of two minutes.

Where the level dey live: na chat template, no be weights

Thinking model dey trained to first output reasoning segment, usually dem wrap am with <think> and </think> tags, before e give final answer. Effort level na instruction wey the model chat template dey write inside prompt. That template na Jinja file wey dey ship with the model. E dey read variable like reasoning_effort and render different system-level line for each value. The model train to make e scratchpad shorter or longer based on that line.

Two things follow from this. Level names belong to the model, no be your runtime. So name from one model card fit mean nothing for another model. And if anything for the chain replace the model chat template with generic one, dem no go render the variable, and the setting go silently do nothing.

As of 2026-08-20, Qwen3.8-27B model card document three effort levels: low, medium and xhigh, with xhigh as default. No high dey. You fit switch thinking with enable_thinking, and e dey on by default. The card also document preserve_thinking, wey dey on by default, to keep reasoning from earlier turns inside conversation history. gpt-oss dey use low, medium and high instead. Plenty other model families accept only boolean value. Read the card for the exact version wey you pull, because these names no be standard. How to get 27B model running on VPS come first. This page na about wetin to set after e answer.

Why high effort dey cost more for a VPS

Output tokens. Every token wey system generate dey use work. Dem dey generate reasoning tokens with the same decode process wey dem use for answer tokens. Suppose task produce 200 answer tokens and 4,000 reasoning tokens. You generate 4,200 tokens, but reader see only 200. Memory bandwidth and the quantisation wey you choose determine your decode rate, so the only thing wey remain to control na the token count.

Wall clock. Person dey wait for the first token of the answer, because everything before that na blank screen or collapsed spinner. Reasoning dey come first, so the wait roughly equal reasoning token count divided by your decode rate, plus prompt processing. If reasoning length double, that wait go double too.

Context. Reasoning tokens occupy context window like every other token. With preserve_thinking on, scratchpad from turn one still dey inside prompt for turn five. So prompt processing dey slower every turn while the window dey fill from both ends. If you raise num_ctx to hold am, e go use KV cache memory. For VPS wey no get GPU, na system RAM be this, and you fit no get spare RAM.

When to raise the level, and when to leave am low

Raise am for work wey wrong intermediate step fit spoil the result: multi-step arithmetic and unit conversion, planning edit across several files, code wey must compile, and constraint problems wey one answer must satisfy several conditions at once. For these ones, scratchpad dey do real work, and longer one na cheap way to catch error wey model for otherwise commit.

Leave am low when answer don dey inside input and na to move am be the work. Extraction, classification, tagging, translation, rewriting, summarising and formatting all dey here. The reasoning segment mostly dey repeat the task, and e dey give model room to argue itself comot from correct first instinct.

Leave am low for anything interactive too. For chat box or editor, you dey inside the loop, so fast answer wey you fit correct better pass slow one wey you must wait for. Na this be the real trade-off behind pointing coding agent at local model: agent dey make plenty small calls, and reasoning tax dey apply to every one of dem.

How to set the level in llama.cpp

llama.cpp dey write the variable directly inside the template. This make e be the runtime wey you fit confirm say the level reach. Point -m to the GGUF wey you already get.

llama-server -m ./qwen3.8-27b-Q4_K_M.gguf \
  --jinja \
  --reasoning-effort medium \
  --reasoning-format deepseek \
  -c 32768 \
  --host 127.0.0.1 --port 8080

--jinja dey use the model own chat template, and current builds enable am by default. --reasoning-effort accepts default, minimal, low, medium, high, xhigh or max. default mean make e leave the template own default unchanged. This list na llama.cpp vocabulary, no be the model own vocabulary. So pass only name wey the card list: if the template no define the level, request fit fail with template error. --reasoning-format deepseek dey move the reasoning comot from message.content put inside message.reasoning_content. Na this one make the split measurable for the next section.

If you wan turn thinking off instead of shortening am, set the template variable by yourself:

llama-server -m ./qwen3.8-27b-Q4_K_M.gguf --jinja \
  --chat-template-kwargs '{"enable_thinking": false}'

--reasoning-budget na different mechanism. E dey cap the reasoning segment by token count. 0 dey end am immediately, while -1 leave am without limit. E no dey ask the model to plan a shorter reasoning segment. Both flags apply to the whole server. llama-server no accept reasoning_effort as per-request field. So if you wan serve two effort levels at the same time, you need two processes for two ports.

vLLM exposes the same variable for each request, inside the OpenAI-compatible body:

{"model": "Qwen/Qwen3.8-27B",
 "messages": [{"role": "user", "content": "Summarise this changelog in two lines."}],
 "chat_template_kwargs": {"reasoning_effort": "medium"}}

Ollama dey set level

Ollama get im own field, think, for /api/chat and /api/generate. E dey accept true, false, or one of low, medium, high and max, where max dey ask for the highest level wey the model get. Thinking dey on by default for models wey support am.

ollama run qwen3.8:27b --think=low "Draft a one line commit message for a README typo fix"

{"model": "qwen3.8:27b",
 "messages": [{"role": "user", "content": "Which HTTP status code means the request body was too large?"}],
 "think": "low",
 "stream": false}

The reasoning dey come back for message.thinking and the answer for message.content, already split for you. Inside interactive ollama run session, /set think and /set nothink dey toggle am without restarting.

Now notice the mismatch. Ollama vocabulary na low, medium, high and max. Qwen3.8 template define low, medium and xhigh. Something must map one to the other, and Ollama model carry template wey dem package inside im tag instead of the Jinja file from the original repository. So whether your level reach the model depend on that packaged template. No assume say e work. Measuring am dey take about one minute.

How to measure whether the level really take effect

Send the same prompt for more than one level with temperature at 0, then compare the token counts. For here, jq dey build the body, so you no need escape quotes by hand.

for level in low medium max; do
  body=$(jq -n --arg lvl "$level" '{
    model: "qwen3.8:27b",
    messages: [{role: "user", content: "A pump fills a 4500 litre tank in 25 minutes. A second pump is 40 percent slower. How long do both together take? Answer in minutes."}],
    think: $lvl,
    stream: false,
    options: {temperature: 0, num_ctx: 8192}
  }')
  echo "== $level"
  curl -s http://localhost:11434/api/chat -d "$body" | jq '{
    thinking_chars: (.message.thinking // "" | length),
    answer_chars: (.message.content | length),
    eval_count: .eval_count,
    seconds: (.total_duration / 1e9),
    tok_per_sec: (.eval_count / (.eval_duration / 1e9))
  }'
done

eval_count na every token wey e generate, including reasoning, so the difference between two levels almost entirely na reasoning. thinking_chars dey give you the split directly. Two things suppose happen: the numbers go change between levels, and the answer go remain correct for the lower level. If eval_count dey within normal noise for all three runs, the level no dey apply. The fix na to use runtime wey pass the level, not another level name.

Total time na only half the matter, so measure the time reach the first answer token by streaming and stopping at the first non-empty content chunk. This one need jq and bc.

start=$(date +%s.%N)
curl -sN http://localhost:11434/api/chat -d '{
  "model": "qwen3.8:27b",
  "messages": [{"role": "user", "content": "Explain what a reverse proxy does, in three sentences."}],
  "think": "low",
  "stream": true
}' |
while IFS= read -r line; do
  if [ -n "$(printf '%s' "$line" | jq -r '.message.content // ""')" ]; then
    echo "first answer token after $(echo "$(date +%s.%N) - $start" | bc)s"
    break
  fi
done

Run am for low and run am again for max. The difference na the wait wey you dey pay for. For llama.cpp, the same numbers go return inside the response, and you no need shell arithmetic:

curl -s http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "local", "temperature": 0,
       "messages": [{"role": "user", "content": "A pump fills a 4500 litre tank in 25 minutes. A second pump is 40 percent slower. How long do both together take?"}]}' | jq '{
  reasoning_chars: (.choices[0].message.reasoning_content // "" | length),
  answer_chars: (.choices[0].message.content | length),
  predicted_n: .timings.predicted_n,
  tok_per_sec: .timings.predicted_per_second
}'

Do this for your own box. Somebody measure the published effort comparison for hardware wey no be your own, and your decode rate na the value wey turn token count to seconds. How to measure tokens per second for your own server go give you that value: divide reasoning tokens by your decode rate to get the wait wey you just add.

Wetin fit go wrong

The answer cut off, or content empty while thinking full. Reasoning don use up the generation limit. Ollama num_predict dey cap the whole generation, including reasoning, and reasoning dey happen first. So cap of 512 tokens with high effort fit end the response before the answer start. Ollama reports "done_reason": "length" for that response. Increase the cap or reduce the effort. How num_predict dey count tokens explain the interaction well.

The level no change anything. Token counts dey the same for every level. Either the runtime no dey pass the variable, or the template no dey read am. Check the template wey your runtime dey actually use, instead of the one for the original repository. llama.cpp with --jinja and --chat-template-kwargs dey write the variable by hand, so e good as control: if the level work there but no work anywhere else, the model dey fine and the other runtime dey drop am.

The system reject a level name. If template error happen when you send request, or failure happen for the first message while the server dey otherwise healthy, e usually mean say you pass a level wey the template no define. Example na high for model wey model card list only low, medium and xhigh.

Multi-turn chats dey slow down for every turn. Old reasoning dey remain inside the history. Set preserve_thinking to false if the model support am, or remove the thinking field from the messages wey you send back. If you no do this, prompt processing go grow for every turn while the answers remain the same length.

Quality dey drop for low effort on task wey you think say simple. Some extraction no really be extraction. If the input need unit conversion or rule wey you must apply in order, na reasoning task with short output. Increase the level for that one call instead of increasing am for the whole server.

Run two level at once

llama.cpp dey fix the level when e start, so a box wey dey serve both editor and nightly batch job go need two processes for two ports, and each one get im own --reasoning-effort. Two processes also mean say weights go dey memory two times, unless you separate the jobs by time instead. For one VPS, the cheaper setup normally na low-effort server for anything wey person dey wait for, plus scheduled higher-effort run for work wey nobody dey monitor. Wetin happen when plenty users share one local model apply here too: reasoning tokens na decode work, so when you increase the effort, your effective concurrency go reduce by roughly the same factor wey token count increase.

FAQ

Which default reasoning effort level I suppose use?

Start with the lowest level wey the model get, and increase am only for tasks wey you don watch fail. Some thinking models dey release with high default, and Qwen3.8-27B dey default to xhigh, wey be the highest level, as of August 2026. Dem choose that default to look good for benchmark tables, but benchmark table no dey charge for time. For your own hardware, na seconds you dey pay, so make higher level be something you choose for each task, instead of the setting every request go inherit.

Reasoning tokens dey count against my context window?

Yes. Dem be normal tokens for the output, and dem dey occupy the context window together with everything else. Whether dem go remain there for the next turn depend on the runtime and the model. Qwen3.8 card documents preserve_thinking, wey dey enabled by default. This one keeps earlier reasoning for the history, so long conversation go carry every scratchpad wey e don produce. Set am to false, or remove the thinking field from the messages wey you replay, and prompt processing no go continue to grow.

Why changing the thinking level no dey change my token counts?

The setting no dey reach the chat template. The level na template variable, so e go work only if the runtime passes am and the packaged template reads am. Some runtimes dey release with their own template together with the model, instead of the Jinja file from the original repository. For that case, the variable go drop without any error showing anywhere. Prove am by sending the same prompt at the lowest and highest level, with temperature at 0, then compare eval_count. If the counts match within normal variation, the level dey ignored.

Lower reasoning effort dey make the model less accurate?

E depend on the task, and you suppose measure am instead of assuming. If the answer already dey inside the input, like extraction or rewriting, shorter scratchpad usually no dey change anything. If one intermediate step must correct before the final answer fit correct, like multi-step arithmetic or code wey must compile, accuracy dey reduce when scratchpad shorter. Build a set of twenty prompts from your real workload, run dem at two levels with temperature at 0, then count the wrong answers. That number specific to your workload, and no published table fit give am to you.