SSD Nodes Learn Hosting plans →
How to do am Matt ConnorBy Matt Connor

How to Use Ollama with Your Coding Agent

Connect your coding agent to Ollama with one base URL and any dummy key. See the 11434 endpoints, context-length gotcha, keep-alive, and best local-model jobs.

Wetin you dey connect

You fit use Ollama with your coding agent, and the connection small pass wetin people dey expect. You go change one base URL and choose one model name. The API key field still need value, but the local server dey ignore am, so any string go work.

Ollama dey listen on port 11434 and e dey serve three API surfaces at the same time. /api na Ollama native API. /v1 na the OpenAI-compatible layer, wey cover different formats: /v1/chat/completions, /v1/responses, /v1/completions, /v1/models and /v1/embeddings. Ollama documentation talk say the key dey required there but e dey ignored. The third surface na Anthropic-compatible: the client take http://localhost:11434 as im base URL and append /v1/messages, wey Claude Code dey speak. Most agents dey use /v1/chat/completions, Codex dey use /v1/responses, and Claude Code dey use /v1/messages. Your agent already dey speak one of dem, so nothing else about am go change.

That part fit take five minutes. Whether the result go usable depend on two settings wey almost nobody dey change: the context length and the keep-alive. E also depend on whether you give the model the kind work wey e good at. Each one get im own section, and the honest limits dey for the end.

Which coding agents dey accept a local base URL

The test na one question: does the tool expose a base URL setting? If e do, e fit talk to your server.

Ollama publishes integration pages for Claude Code, OpenCode, Codex, Cline, Roo Code, Zed, JetBrains IDEs and VS Code. Aider documents its own Ollama support separately. That cover most of wetin people mean by coding agent for October 2026. Dem no all dey use the same API shape, and na that difference dey make setups fail.

  • Most agents want an OpenAI-compatible endpoint. Give dem the base URL http://localhost:11434/v1 and any non-empty API key string.
  • Claude Code no accept OpenAI base URL at all. E dey speak Anthropic Messages API, so e need ANTHROPIC_BASE_URL set to http://localhost:11434, where Ollama dey serve /v1/messages.
  • Codex dey use OpenAI Responses API. Ollama dey serve /v1/responses too, and dem add am for version 0.13.3.
  • Agent wey no get base URL setting no fit redirect, because endpoint dey built inside the client. If the agent accept custom URL but need different API shape, put translation layer in front instead, such as self-hosted LiteLLM gateway, then expose your model again in the shape wey the client demand.

Ollama fit write these configs for you. ollama launch opencode starts OpenCode with inline config for the model wey you choose, ollama launch claude do the same for Claude Code, and ollama launch droid --config writes the config without launching the tool.

Install Ollama and pull model wey fit call tools

curl -fsSL https://ollama.com/install.sh | sh
systemctl status ollama --no-pager
ollama pull qwen3-coder:30b
ollama ls

The installer dey add systemd unit and start am, so systemctl status ollama suppose print active (running). If e no do so, journalctl -e -u ollama go print the reason.

The model must support tool calling, because na tool calling agent dey use work. E go read file, write patch, run test, then read the failure and try again. Model wey no fit emit tool call go describe the edit for prose instead of making am, and agent go loop or stop. Look for tools label for the model page on ollama.com before you pull am. qwen3-coder:30b get this label, and as of October 2026 that tag na 19 GB download with 256K context window. If your box na CPU-only or RAM no plenty, the memory arithmetic for the Qwen 27B tag on a VPS go show wetin fit inside 8 to 64 GB before you spend bandwidth download am. Once you pull am, those gigabytes go enter root disk of the server. Na the part of VPS wey usually get the least free space. So where Ollama dey keep its model files and how to move dem elsewhere worth reading before disk full.

Now confirm the names wey the server actually dey serve:

curl http://localhost:11434/v1/models

The strings for that response na wetin your agent config must contain, character for character. If you check am first, e go settle most model-not-found errors. If Ollama never install, the longer walkthrough dey for how to self-host an LLM with Ollama on a VPS.

Point OpenCode go Ollama

Edit ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen3-coder:30b": {
          "name": "qwen3-coder 30b"
        }
      }
    }
  }
}

The key under models na the model name wey OpenCode send go Ollama, so e gats match ollama ls exactly. The name field na only the label wey dey model picker. Start opencode, switch go Ollama provider, and monitor journalctl -e -u ollama to confirm say request reach your server, no be somewhere else. Setup for the agent itself dey covered for running OpenCode for VPS.

Point Claude Code go Ollama

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3-coder:30b

ANTHROPIC_API_KEY dem set to empty string on purpose. Base URL dey control where requests dey go; when you clear the key, e prevent hosted credentials from entering this local setup. ollama launch claude go set all these things for you.

Know wetin the compatibility layer no include. E no implement tool_choice or prompt caching, and e no get token counting endpoint. So, the token numbers wey you see na only approximations from the model own tokenizer. Claude Code still ships with large system prompt and large tool set, so e need more context than chat client. The wider question of wetin carry over and wetin no carry over dey covered for whether you fit self-host Claude.

Make Aider point to Ollama

export OLLAMA_API_BASE=http://127.0.0.1:11434
aider --model ollama_chat/qwen3-coder:30b

Aider documentation dey recommend ollama_chat/ prefix pass ollama/. By default, Aider dey request enough context for each request plus 8k tokens for the reply. E still let you pin context window for each model inside .aider.model.settings.yml, wey useful when one model need different window from the server default:

- name: ollama_chat/qwen3-coder:30b
  extra_params:
    num_ctx: 65536

Why working setup still dey produce nonsense

This na the section wey matter. Ollama dey choose default context length from the total VRAM (video memory for the GPUs) wey e fit see, after e subtract OLLAMA_GPU_OVERHEAD for each GPU wey e detect. The implementation dey use these thresholds:

ChartOllama default context length by aggregate VRAM, checked 2026-10-26
Di data wey dey behind dis chart
[
  {
    "label": "Under 23 GiB aggregate VRAM",
    "default_context_tokens": "4,096"
  },
  {
    "label": "23 to under 47 GiB aggregate VRAM",
    "default_context_tokens": "32,768"
  },
  {
    "label": "47 GiB aggregate VRAM or more",
    "default_context_tokens": "262,144"
  }
]

Most VPS plans, and every CPU-only server, dey fall for the first row: 4,096 tokens. Multiple smaller GPUs fit join together reach the 47 GiB threshold for the 262,144 tokens wey dey the last row.

Agent dey pass 4096 tokens before e even start work. System prompt, tool definitions, repository listing, and the first file wey e open don already pass that size. Na here the main problem dey happen: nothing dey show error. Aider documentation talk say Ollama dey silently discard context wey pass the window. The oldest tokens dey fall out, so model fit confidently answer about file wey e no fit see again, or forget instruction wey you give am two steps earlier. Na this mechanism dey cause most reports say local model too dumb to write code. Choosing the number itself na another decision, and how much num_ctx dey cost for KV cache memory at each size worth reading before you settle for one.

Ollama documentation talk say tasks like agents and coding tools suppose use at least 64000 tokens. Set am for the server:

sudo systemctl edit ollama.service

Add these lines inside the override file:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama
ollama ps

Send request from your agent after the restart, then run ollama ps again so one loaded model dey available to inspect. ollama ps na the check. E go print one CONTEXT column, and that number na wetin model actually receive. Your ID and SIZE go differ:

NAME               ID              SIZE     PROCESSOR    CONTEXT    RUNNER    UNTIL
qwen3-coder:30b    a1b2c3d4e5f6    24 GB    100% GPU     64000      ggml      4 minutes from now

Set am for the server instead of inside the agent for two reasons. OpenAI chat completions schema no get field for context length, so OpenAI-compatible client no fit request one. Also, server default dey apply to every agent unless model or request override am. Output side get its own limit, and unlike context length, e dey pass through the compatibility endpoint. So num_predict and the max_tokens field wey maps to am na wetin you suppose use when reply stop halfway through a patch. If one model need different window, bake am into copy with a Modelfile:

FROM qwen3-coder:30b
PARAMETER num_ctx 65536
ollama create qwen3-coder-64k -f Modelfile

Context no free. Longer window dey cost more memory, so monitor the PROCESSOR column. 100% GPU na wetin you want. Once part of the model spill enter CPU, token rate go fall reach point wey agent loop no fit work again, and how to measure tokens per second for local LLM na how you go find the real limit for your machine. how much RAM and CPU coding agent VPS need cover how to size the machine before you buy am.

Keep model loaded between requests

By default, Ollama dey unload model 5 minutes after e receive the last request. This fit work for chat box, but e no good for agent work. You pause to read a diff, timer expire, and the next request load tens of gigabytes of weights from disk again before the first token show. E go look like say the system hang.

OLLAMA_KEEP_ALIVE dey take duration string like 10m or 24h, plain number of seconds, -1 to keep model loaded forever, or 0 to unload am immediately. Set am beside the context length:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"
Environment="OLLAMA_KEEP_ALIVE=-1"

The keep_alive request field dey available for Ollama native /api/generate and /api/chat endpoints, and as Ollama extension for /v1/chat/completions. Agent wey fit expose extra request fields fit send am there; otherwise use the environment variable. When you need the memory back, ollama stop qwen3-coder:30b go unload the model without stopping the server. If you want the setting to remain after reboot, or you want compare keeping the weights for memory all day with getting that memory back, keeping Ollama model loaded for memory fit handle both.

Running Ollama for separate server

Ollama dey bind to localhost. To reach am from another machine, set OLLAMA_HOST=0.0.0.0:11434 for the same systemd override, then restart the service.

Do this only for private network. Ollama documentation talk say local API no require authentication, so if port 11434 open to internet, anybody fit use your hardware. Two safe options dey. Keep the bind for localhost and forward the port over SSH from your laptop:

ssh -N -L 11434:localhost:11434 you@your-vps

Your agent go still point to http://localhost:11434/v1 and e no go know say anything change. The other option na VPN, with Ollama bound to the VPN address instead of 0.0.0.0. If several people or several agents go share one box, Ollama fit handle parallel requests when memory allow am, and the comparison between Ollama and vLLM show where the throughput difference start to cause problem.

Where local coding model dey win, and where e no dey

Agent wey model you host dey drive no fit replace frontier API for every task. E dey win clearly for four kinds of work.

  • Bulk mechanical edits, where each change small and you fit check am. Rename things across repository, add type hints, write docstrings, translate comments. Model fit run for hours and bill no go change.
  • Work wey must not leave your hardware. Client code under confidentiality agreement, or internal repository wey you no get permission to send to third party.
  • Offline and air-gapped machines, where hosted API no dey available to call at all.
  • Predictable cost. Once you don pay for server, agent wey dey burn tokens inside loop no cost anything extra. This na opposite of metered API. Where GPU VPS dey break even against API tokens get the arithmetic.

E dey lose for long multi-step tasks. "Find why this test dey fail, fix the cause, update the callers" need many correct tool calls one after another, while the complete history still dey for context. Model for 8B to 14B range on modest server fit produce malformed tool call, or lose the plan after few turns. Then you go spend more time directing am pass the time wey the task suppose take. This no be prompt problem wey you fit solve just by writing better prompt. Na capacity problem.

E still dey lose anytime mistake cost plenty and you no go read every line. Give local model narrow jobs wey you go verify the output, and keep hosted model for work wey you no go check step by step.

Failure modes, and the strings wey you go see

curl: (7) Failed to connect to localhost port 11434 after 0 ms: Connection refused. The server no dey run, or the agent dey point to another host. Run systemctl status ollama, then journalctl -e -u ollama.

The agent dey report say the model no exist. The name for your config no match any name wey the server dey serve. Compare am with curl http://localhost:11434/v1/models and copy the string from there. The tag dey part of the name, so config wey name a tag you never pull go fail even though similar model dey installed.

The agent dey answer with prose and e never edit any file. Either the model no support tools, or the request plus the tool definitions don already fill the context window. Check the tools label for the model page, then check the CONTEXT column for ollama ps.

Long silence before the first token, then normal speed. The keep-alive don expire and the weights dey read from disk again. Set OLLAMA_KEEP_ALIVE.

The model dey contradict file wey e just read. Na context truncation. ollama ps usually dey show a CONTEXT value wey small pass wetin you think say you set, because the environment variable enter your shell instead of the systemd unit.

Everything dey work, but slow, and PROCESSOR dey show 100% CPU or a CPU/GPU split. Usually, the model plus its context no fit inside VRAM. If PROCESSOR read 100% CPU for machine wey get GPU, rule out the other cause first: Ollama fit never find the GPU at all. Ollama documentation note say for Linux, after suspend/resume cycle, e sometimes fail to discover NVIDIA GPU and fall back to CPU, and reloading the driver with sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm dey work around am. If the GPU dey found and the split still dey, reduce the context length, or move to smaller model or smaller quantisation. Before you re-pull, wetin q4_K_M, q8_0 and fp16 each cost for memory, and where the quality really drop go tell you how much room one step down go give you and wetin you go sacrifice. Unknown fit also be a possible PROCESSOR value; e no prove say CPU offloading dey happen.

FAQ

I fit point Claude Code go Ollama?

Yes, but e no work with OpenAI-compatible URL. Claude Code dey speak Anthropic Messages API, and Ollama dey serve that format for /v1/messages on the same port 11434. Export ANTHROPIC_BASE_URL=http://localhost:11434, ANTHROPIC_AUTH_TOKEN=ollama and empty ANTHROPIC_API_KEY, then start am with claude --model qwen3-coder:30b. ollama launch claude go write the same settings for you. The compatibility layer no implement tool_choice or prompt caching, and e no get token counting endpoint, so the token counts wey e report na approximations.

Why my local model dey answer about code wey e no fit see?

Because the request don pass the context window, and the oldest part drop without any error. Ollama dey set the default context from the total detected VRAM after e subtract the configured overhead for each GPU: below 23 GiB the default na 4,096 tokens, from 23 to under 47 GiB e na 32,768 tokens, and for 47 GiB or more e na 262,144 tokens. Set OLLAMA_CONTEXT_LENGTH=64000 inside the systemd unit, restart Ollama, and confirm the CONTEXT column for ollama ps show the new value.

Which model I suppose run for coding agent on VPS?

Pick the biggest model wey get tools label and still fit inside memory with 64k context window, and prefer one wey dem tune for code. qwen3-coder:30b na the common answer for GPU server wey get enough VRAM. If that tag too big for your server, the RAM figures and CPU-only speeds for Nemotron 3.5 Lightning fit give useful comparison before you commit to download. Below roughly 14B parameters, model fit still answer code questions well and still fail for multi-step edits, because agent work dey punish small formatting mistakes inside tool calls. Test am with one real task from your own repository instead of sample prompt.

I need GPU to run coding agent on my own model?

For real use, yes. CPU-only inference dey work and e okay for single questions, but agent dey send plenty requests for each task and each one dey read long history again, so slow token rate fit turn two-minute task to one hour. Check the PROCESSOR column for ollama ps: 100% CPU or CPU/GPU split mean say CPU dey used, and that one dey slow inference. Unknown no prove say CPU offloading dey happen.

Dis article help you?

#ollama#coding-agent#openai-compatible#local-llm#self-hosted-ai