How to Use Ollama With Your Coding Agent

Wetin you dey connect

You fit use Ollama with your coding agent, and the connection small pass wetin people dey expect. You go change one base URL and choose one model name. The API key field still need value, but the local server dey ignore am, so any string go work.

Ollama dey listen for port 11434 and dey serve two request shapes at the same time. /v1/chat/completions na the OpenAI-compatible shape, and Ollama documentation talk say the key for there na required but ignored. /v1/messages na the Anthropic-compatible shape, wey Claude Code dey speak. Your agent already dey speak one of the two, so nothing else about am go change.

This part fit take five minutes. Whether the result go useful depend on two settings wey almost nobody dey change, the context length and the keep-alive, plus whether you give the model the kind work wey e good at. Each one get im own section, and the honest limits dey for the end.

Wetin coding agents dey accept local base URL

The test na one question: does the tool expose a base URL setting? If e get am, e fit talk to your server.

Ollama publish integration pages for Claude Code, OpenCode, Codex, Cline, Roo Code, Zed, JetBrains IDEs and VS Code. Aider document im own Ollama support separately. This cover most of wetin people mean by coding agent for August 2026. Dem no all dey use the same format, and na this difference dey make setups fail.

• Most agents want OpenAI-compatible endpoint. Give dem the base URL http://localhost:11434/v1 and any non-empty API key string.
• Claude Code no accept OpenAI base URL at all. E dey use Anthropic Messages API, so you need set ANTHROPIC_BASE_URL to http://localhost:11434, where Ollama dey serve /v1/messages.
• Codex dey use OpenAI Responses API. Ollama dey serve /v1/responses too, and dem add am for version 0.13.3.
• Agent wey no get base URL setting no fit redirect, because the endpoint dey built inside the client. Put translation layer for front instead, like self-hosted LiteLLM gateway, then expose your model again for the format wey the client require.

Ollama fit write these configs for you. ollama launch opencode dey start OpenCode with inline config for the model wey you choose, ollama launch claude do the same thing for Claude Code, and ollama launch droid --config write the config without launching the tool.

Install Ollama and pull model wey fit call tools

curl -fsSL https://ollama.com/install.sh | sh
systemctl status ollama --no-pager
ollama pull qwen3-coder:30b
ollama ls

The installer dey add systemd unit and start am, so systemctl status ollama suppose print active (running). If e no do so, journalctl -e -u ollama go print the reason.

The model must support tool calling, because na tool calling agent dey use work. E go read file, write patch, run test, then read the failure and try again. Model wey no fit emit tool call go describe the edit for prose instead of making am, and the agent go dey loop or stop. Look for tools label for the model page on ollama.com before you pull am. qwen3-coder:30b carry am, and as of August 2026 that tag na 19 GB download with 256K context window. If your box na CPU-only or RAM no plenty, the memory arithmetic for the Qwen 27B tag on a VPS show wetin fit enter for 8 to 64 GB before you spend the download.

Now confirm which names the server dey actually serve:

curl http://localhost:11434/v1/models

The strings for that response na wetin your agent config must contain, character for character. If you check am first, e go settle most model-not-found errors. If Ollama never install, the longer walkthrough dey for how to self-host LLM with Ollama for VPS.

Point OpenCode go Ollama

Edit ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen3-coder:30b": {
          "name": "qwen3-coder 30b"
        }
      }
    }
  }
}

The key under models na the model name wey dem send go Ollama, so e gats match ollama ls exactly. The name field na only the label for model picker. Start opencode, switch go Ollama provider, and monitor journalctl -e -u ollama to confirm say request reach your server instead of somewhere else. Setup for the agent itself dey covered for running OpenCode for a VPS.

Claude Code point go Ollama

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3-coder:30b

ANTHROPIC_API_KEY set to empty string on purpose. If real key dey inside environment, e go send your requests go hosted API instead. That one fit cause bill, and no local inference go happen. ollama launch claude set everything for you.

Know wetin compatibility layer no include. E no implement tool_choice or prompt caching. E no get token counting endpoint too, so the token numbers wey you see na only approximations from the model own tokenizer. Claude Code dey also ship with large system prompt and large tool set, so e need more context than chat client. The bigger question of wetin carry over and wetin no carry over dey covered for whether you fit self-host Claude.

Point Aider go Ollama

export OLLAMA_API_BASE=http://127.0.0.1:11434
aider --model ollama_chat/qwen3-coder:30b

Aider documentation recommend ollama_chat/ prefix pass ollama/. E also let you set context window for each model inside .aider.model.settings.yml. This one useful when one model need different window from the server default:

- name: ollama_chat/qwen3-coder:30b
  extra_params:
    num_ctx: 65536

Why working setup still dey produce nonsense

Na this section matter. Ollama dey choose default context length from the VRAM (video memory for GPU) wey e fit see, and dem don publish those defaults:

[
  {
    "label": "Under 24 GiB VRAM",
    "default_context_tokens": "4,096"
  },
  {
    "label": "24 to 48 GiB VRAM",
    "default_context_tokens": "32,768"
  },
  {
    "label": "48 GiB VRAM or more",
    "default_context_tokens": "262,144"
  }
]

Most VPS plans, and every CPU-only server, dey enter the first row: 4,096 tokens. Na only large GPU get the 262,144 tokens for the last row.

Agent dey use 4096 tokens before e even start work. The system prompt, tool definitions, repository listing, and the first file wey e open don already pass that size. Na here the main problem start: no error dey show. Aider documentation talk say Ollama dey silently discard context wey pass the window size. The oldest tokens dey fall out, so model fit confidently answer about file wey e no fit see again, or forget instruction wey you give am two steps before. Na this mechanism dey cause most reports say local model too dull to write code.

Ollama documentation talk say tasks like agents and coding tools suppose use at least 64000 tokens. Set am for the server:

sudo systemctl edit ollama.service

Add these lines for the override file:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama
ollama ps

ollama ps na the check. E dey print a CONTEXT column, and that number na wetin model actually receive. Your ID and SIZE go differ:

NAME               ID              SIZE     PROCESSOR    CONTEXT    UNTIL
qwen3-coder:30b    a1b2c3d4e5f6    24 GB    100% GPU     64000      4 minutes from now

Set am for the server instead of inside the agent for two reasons. OpenAI chat completions schema no get field for context length, so OpenAI-compatible client no fit request one. And the setting dey apply per server, so every agent wey you point to the box go inherit am. If one model need different window, bake am into a copy with a Modelfile:

FROM qwen3-coder:30b
PARAMETER num_ctx 65536

ollama create qwen3-coder-64k -f Modelfile

Context no free. Longer window dey cost more memory, so monitor the PROCESSOR column. 100% GPU na wetin you want. Once part of the model spill enter CPU, token rate go drop reach point wey agent loop no fit work again, and how to measure tokens per second for local LLM na how you go find the real ceiling for your box. How to size the machine before you buy am dey covered for how much RAM and CPU coding agent VPS need.

Model loaded dey between requests

By default Ollama dey unload model 5 minutes after its last request. This dey okay for chat box but e no good for agent work. You pause to read diff, timer run out, and the next request reloads tens of gigabytes of weights from disk before the first token show. E go look like hang.

OLLAMA_KEEP_ALIVE dey take duration string like 10m or 24h, plain number of seconds, -1 to keep model loaded forever, or 0 to unload am immediately. Set am beside context length:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"
Environment="OLLAMA_KEEP_ALIVE=-1"

The keep_alive request field dey only for Ollama native /api/generate and /api/chat endpoints, no be for compatibility endpoints. So agent no fit set am per request. The environment variable na the only control wey you get. When you need the memory back, ollama stop qwen3-coder:30b go unload the model without stopping the server.

Ollama run for separate server

Ollama dey bind to localhost. To reach am from another machine, set OLLAMA_HOST=0.0.0.0:11434 for the same systemd override, then restart the service.

Do this only for private network. Ollama documentation talk say local API no need authentication, so if port 11434 open to internet, anybody fit use your hardware and read anything wey your agent send. Two safe options dey. Keep the bind for localhost and forward the port through SSH from your laptop:

ssh -N -L 11434:localhost:11434 you@your-vps

Your agent still dey point to http://localhost:11434/v1 and e no go know any difference. The other option na VPN, with Ollama bound to the VPN address instead of 0.0.0.0. If several people or several agents go share one box, Ollama scheduler no dey designed for that kind load, and the comparison between Ollama and vLLM show where the throughput difference start to cause problem.

Where local coding model dey win, and where e no dey

Agent wey model you host dey drive no replace frontier API for every task. E dey win clearly for four kinds of work.

• Bulk mechanical edits, where every change small and you fit check am. Rename things across repository, add type hints, write docstrings, or translate comments. The model fit run for hours and the bill no go increase.
• Work wey no suppose leave your hardware. Client code under confidentiality agreement, or internal repository wey you no get permission to send to third party.
• Offline and air-gapped machines, where hosted API no dey available to call at all.
• Predictable cost. Once you don pay for the server, agent wey dey burn tokens inside loop no cost extra, unlike metered API. Where GPU VPS fit break even against API tokens get the calculation.

E no do well for long multi-step tasks. “Find why this test fails, fix the cause, update the callers” need many correct tool calls one after another, while the full history still dey inside context. Model for the 8B to 14B range on modest server fit produce malformed tool call, or forget the plan after some turns. Then you go spend more time directing am than the task suppose take. This no be prompt problem wey you fit solve by writing more prompt. Na capacity problem.

E still no do well whenever mistake dey expensive and you no go read every line. Give local model narrow jobs wey you go verify, and keep hosted model for work wey you no go check step by step.

Wahala wey fit happen, and strings wey you go see

curl: (7) Failed to connect to localhost port 11434 after 0 ms: Connection refused. The server no dey run, or agent dey point to another host. Run systemctl status ollama, then journalctl -e -u ollama.

Agent dey report say model no dey exist. Name wey dey your config no match any name wey server dey serve. Compare am with curl http://localhost:11434/v1/models and copy the string from there. Tag na part of the name, so config wey name tag wey you never pull go fail even though similar model dey installed.

Agent dey answer with prose and e never edit file. Either model no get tool support, or request plus tool definitions don already fill context window. Check tools label for model page, then check CONTEXT column for ollama ps.

Long silence before first token, then normal speed. Keep-alive don expire and weights dey read from disk again. Set OLLAMA_KEEP_ALIVE.

Model dey contradict file wey e just read. Context truncation. ollama ps usually dey show CONTEXT value wey small pass wetin you think say you set, because environment variable go your shell instead of the systemd unit.

Everything dey work, but slowly, and PROCESSOR no be 100% GPU. Model plus its context no fit inside VRAM. Reduce context length, or move to smaller model or smaller quantisation.

FAQ

I fit point Claude Code go Ollama?

Yes, but no be with OpenAI-compatible URL. Claude Code dey use Anthropic Messages API, and Ollama dey serve that format for /v1/messages on the same port 11434. Export ANTHROPIC_BASE_URL=http://localhost:11434, ANTHROPIC_AUTH_TOKEN=ollama and empty ANTHROPIC_API_KEY, then start am with claude --model qwen3-coder:30b. ollama launch claude go write the same settings for you. This compatibility layer no implement tool_choice or prompt caching, and e no get token counting endpoint, so reported token counts na approximation.

Why my local model dey answer about code wey e no fit see?

Because the request don pass the context window, and the oldest part drop without error. Ollama dey set default context from the VRAM wey e find, and below 24 GiB that default na 4,096 tokens. Agent system prompt and tool definitions alone fit pass this limit. Set OLLAMA_CONTEXT_LENGTH=64000 for the systemd unit, restart Ollama, then confirm say CONTEXT column for ollama ps dey show the new value.

Which model I suppose run for coding agent on VPS?

Choose the biggest model wey get tools label and still fit inside memory with 64k context window. Prefer model wey dem tune for code. qwen3-coder:30b na the common answer for GPU server wey get enough VRAM. Below roughly 14B parameters, model fit still answer questions about code well but fail for multi-step edits, because agent work dey punish small formatting mistakes for tool calls. Test am with one real task from your own repository instead of sample prompt.

I need GPU to run coding agent on my own model?

For practical use, yes. CPU-only inference dey work and e dey okay for single questions, but agent dey send plenty requests for each task and each request dey read long history again. So slow token rate fit turn two-minute task to one hour. Check PROCESSOR column for ollama ps: any value wey no be 100% GPU mean say part of the model dey run on CPU, and token rate go drop sharply.