Use Ollama with your coding agent

What you are connecting

You can use Ollama with your coding agent, and the connection is smaller than people expect. You change one base URL and you pick one model name. The API key field still wants a value, but the local server ignores it, so any string works.

Ollama listens on port 11434 and serves two request shapes at the same time. /v1/chat/completions is the OpenAI-compatible shape, and Ollama's documentation describes the key there as required but ignored. /v1/messages is the Anthropic-compatible shape, which is what Claude Code speaks. Your agent already speaks one of the two, so nothing else about it changes.

That part takes five minutes. Whether the result is usable comes down to two settings almost nobody changes, the context length and the keep-alive, and to giving the model the kind of work it is good at. Both get their own section, and the honest limits are at the end.

Which coding agents accept a local base URL

The test is one question: does the tool expose a base URL setting? If it does, it can talk to your server.

Ollama publishes integration pages for Claude Code, OpenCode, Codex, Cline, Roo Code, Zed, JetBrains IDEs and VS Code. Aider documents its own Ollama support separately. That covers most of what people mean by a coding agent in August 2026. They do not all speak the same shape, and that difference is where setups fail.

• Most agents want an OpenAI-compatible endpoint. Give them the base URL http://localhost:11434/v1 and any non-empty API key string.
• Claude Code does not accept an OpenAI base URL at all. It speaks the Anthropic Messages API, so it needs ANTHROPIC_BASE_URL set to http://localhost:11434, where Ollama serves /v1/messages.
• Codex uses the OpenAI Responses API. Ollama serves /v1/responses as well, added in version 0.13.3.
• An agent with no base URL setting cannot be redirected, because the endpoint is built into the client. Put a translation layer in front instead, such as a self-hosted LiteLLM gateway, and re-expose your model in whatever shape the client demands.

Ollama can write these configs for you. ollama launch opencode starts OpenCode with an inline config for the model you choose, ollama launch claude does the same for Claude Code, and ollama launch droid --config writes the config without launching the tool.

Install Ollama and pull a model that can call tools

curl -fsSL https://ollama.com/install.sh | sh
systemctl status ollama --no-pager
ollama pull qwen3-coder:30b
ollama ls

The installer adds a systemd unit and starts it, so systemctl status ollama should print active (running). If it does not, journalctl -e -u ollama prints the reason.

The model must support tool calling, because tool calling is how an agent works. It reads a file, writes a patch, runs the test, then reads the failure and tries again. A model that cannot emit a tool call will describe the edit in prose instead of making it, and the agent will loop or stop. Look for the tools label on the model's page on ollama.com before you pull. qwen3-coder:30b carries it, and as of August 2026 that tag is a 19 GB download with a 256K context window.

Now confirm which names the server actually serves:

curl http://localhost:11434/v1/models

The strings in that response are what your agent config must contain, character for character. Checking it first settles most model-not-found errors. If Ollama is not installed yet, the longer walkthrough is in self-hosting an LLM with Ollama on a VPS.

Point OpenCode at Ollama

Edit ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "qwen3-coder:30b": {
          "name": "qwen3-coder 30b"
        }
      }
    }
  }
}

The key under models is the model name sent to Ollama, so it has to match ollama ls exactly. The name field is only the label in the model picker. Start opencode, switch to the Ollama provider, and watch journalctl -e -u ollama to confirm the request arrived at your server rather than somewhere else. Setting up the agent itself is covered in running OpenCode on a VPS.

Point Claude Code at Ollama

export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=""
export ANTHROPIC_BASE_URL=http://localhost:11434
claude --model qwen3-coder:30b

ANTHROPIC_API_KEY is set to an empty string on purpose. A real key left in the environment sends your requests to the hosted API instead, so you get a bill and no local inference. ollama launch claude sets all of this for you.

Know what the compatibility layer leaves out. It does not implement tool_choice or prompt caching, and it has no token counting endpoint, so the token numbers you see are approximations from the model's own tokenizer. Claude Code also ships a large system prompt and a large tool set, so it needs more context than a chat client does. The wider question of what carries over and what does not is covered in whether you can self-host Claude.

Point Aider at Ollama

export OLLAMA_API_BASE=http://127.0.0.1:11434
aider --model ollama_chat/qwen3-coder:30b

Aider's documentation recommends the ollama_chat/ prefix over ollama/. It also lets you pin the context window per model in .aider.model.settings.yml, which is useful when one model needs a different window from the server default:

- name: ollama_chat/qwen3-coder:30b
  extra_params:
    num_ctx: 65536

Why a working setup still produces nonsense

This is the section that matters. Ollama chooses a default context length from the VRAM (video memory on the GPU) it can see, and those defaults are published:

[
  {
    "label": "Under 24 GiB VRAM",
    "default_context_tokens": "4,096"
  },
  {
    "label": "24 to 48 GiB VRAM",
    "default_context_tokens": "32,768"
  },
  {
    "label": "48 GiB VRAM or more",
    "default_context_tokens": "262,144"
  }
]

Most VPS plans, and every CPU-only server, land in the first row: 4,096 tokens. Only a large GPU gets the 262,144 tokens in the last row.

An agent passes 4096 tokens before it does any work. The system prompt, the tool definitions, the repository listing and the first file it opens are already larger than that. What happens next is the whole problem: nothing errors. Aider's documentation states that Ollama silently discards context that exceeds the window. The oldest tokens fall out, so the model answers confidently about a file it can no longer see, or forgets an instruction you gave two steps earlier. That mechanism is behind most reports that a local model is too dumb to write code.

Ollama's documentation says tasks like agents and coding tools should be set to at least 64000 tokens. Set it on the server:

sudo systemctl edit ollama.service

Add these lines in the override file:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama
ollama ps

ollama ps is the check. It prints a CONTEXT column, and that number is what the model actually received. Your ID and SIZE will differ:

NAME               ID              SIZE     PROCESSOR    CONTEXT    UNTIL
qwen3-coder:30b    a1b2c3d4e5f6    24 GB    100% GPU     64000      4 minutes from now

Set it on the server rather than in the agent, for two reasons. The OpenAI chat completions schema has no field for context length, so an OpenAI-compatible client cannot ask for one. And the setting is per server, so every agent you point at the box inherits it. If one model needs a different window, bake it into a copy with a Modelfile:

FROM qwen3-coder:30b
PARAMETER num_ctx 65536

ollama create qwen3-coder-64k -f Modelfile

Context is not free. A longer window costs more memory, so watch the PROCESSOR column. 100% GPU is what you want. Once part of the model spills to the CPU the token rate falls far enough that an agent loop becomes unusable, and measuring tokens per second on a local LLM is how you find the real ceiling for your box. Sizing the machine before you buy it is covered in how much RAM and CPU a coding agent VPS needs.

Keep the model loaded between requests

By default Ollama unloads a model 5 minutes after its last request. That is right for a chat box and wrong for agent work. You pause to read a diff, the timer runs out, and the next request reloads tens of gigabytes of weights from disk before the first token appears. It reads as a hang.

OLLAMA_KEEP_ALIVE takes a duration string such as 10m or 24h, a plain number of seconds, -1 to keep the model loaded indefinitely, or 0 to unload immediately. Set it beside the context length:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=64000"
Environment="OLLAMA_KEEP_ALIVE=-1"

The keep_alive request field exists only on Ollama's native /api/generate and /api/chat endpoints, not on the compatibility endpoints, so an agent cannot set it per request. The environment variable is the only lever you have. When you need the memory back, ollama stop qwen3-coder:30b unloads the model without stopping the server.

Running Ollama on a separate server

Ollama binds to localhost. To reach it from another machine, set OLLAMA_HOST=0.0.0.0:11434 in the same systemd override and restart the service.

Do that only on a private network. Ollama's documentation states that no authentication is required for the local API, so port 11434 open to the internet means anyone can use your hardware and read whatever your agent sends. Two safe options. Keep the bind on localhost and forward the port over SSH from your laptop:

ssh -N -L 11434:localhost:11434 you@your-vps

Your agent keeps pointing at http://localhost:11434/v1 and never knows the difference. The other option is a VPN, with Ollama bound to the VPN address instead of 0.0.0.0. If several people or several agents will share one box, Ollama's scheduler is not built for that load, and the comparison between Ollama and vLLM shows where the throughput difference starts to hurt.

Where a local coding model wins, and where it does not

An agent driven by a model you host does not replace a frontier API on every task. It wins clearly on four kinds of work.

• Bulk mechanical edits, where each change is small and you can check it. Renaming across a repository, adding type hints, writing docstrings, translating comments. The model runs for hours and the bill does not move.
• Work that must not leave your hardware. Client code under a confidentiality agreement, or an internal repository you are not allowed to send to a third party.
• Offline and air-gapped machines, where there is no hosted API to call at all.
• Predictable cost. Once the server is paid for, an agent that burns tokens in a loop costs nothing extra, which is the opposite of a metered API. Where a GPU VPS breaks even against API tokens has the arithmetic.

It loses on long multi-step tasks. "Find why this test fails, fix the cause, update the callers" needs many correct tool calls in a row, with the whole history still in context. A model in the 8B to 14B range on a modest server will produce a malformed tool call, or lose the plan after a few turns, and you spend more time steering it than the task would have taken. That is not a prompt problem you can write your way out of. It is capacity.

It also loses whenever being wrong is expensive and you will not read every line. Give the local model narrow jobs whose output you verify, and keep a hosted model for the work you would not check step by step.

Failure modes, and the strings you will see

curl: (7) Failed to connect to localhost port 11434 after 0 ms: Connection refused. The server is not running, or the agent is pointed at a different host. Run systemctl status ollama, then journalctl -e -u ollama.

The agent reports that the model does not exist. The name in your config does not match a name the server serves. Compare it against curl http://localhost:11434/v1/models and copy the string from there. The tag is part of the name, so a config naming a tag you never pulled fails even though a similar model is installed.

The agent answers in prose and never edits a file. Either the model has no tool support, or the request plus its tool definitions already fills the context window. Check the tools label on the model page, then check the CONTEXT column in ollama ps.

A long silence before the first token, then normal speed. The keep-alive expired and the weights are being read from disk again. Set OLLAMA_KEEP_ALIVE.

The model contradicts a file it just read. Context truncation. ollama ps usually shows a CONTEXT value smaller than you think you set, because the environment variable went to your shell instead of to the systemd unit.

Everything works, slowly, and PROCESSOR is not 100% GPU. The model plus its context does not fit in VRAM. Lower the context length, or move to a smaller model or a smaller quantisation.

FAQ

Can I point Claude Code at Ollama?

Yes, but not with an OpenAI-compatible URL. Claude Code speaks the Anthropic Messages API, and Ollama serves that shape at /v1/messages on the same port 11434. Export ANTHROPIC_BASE_URL=http://localhost:11434, ANTHROPIC_AUTH_TOKEN=ollama and an empty ANTHROPIC_API_KEY, then start it with claude --model qwen3-coder:30b. ollama launch claude writes the same settings for you. The compatibility layer does not implement tool_choice or prompt caching, and it has no token counting endpoint, so reported token counts are approximations.

Why does my local model answer about code it cannot see?

Because the request no longer fits the context window, and the oldest part of it was dropped with no error. Ollama sets its default context from the VRAM it finds, and below 24 GiB that default is 4,096 tokens, which an agent's system prompt and tool definitions exceed on their own. Set OLLAMA_CONTEXT_LENGTH=64000 in the systemd unit, restart Ollama, and confirm the CONTEXT column in ollama ps shows the new value.

Which model should I run for a coding agent on a VPS?

Pick the largest model carrying a tools label that still fits in memory with a 64k context window, and prefer one tuned for code. qwen3-coder:30b is the common answer on a GPU server with enough VRAM. Below roughly 14B parameters a model can still answer questions about code well and still fail at multi-step edits, because agent work punishes small formatting mistakes in tool calls. Test with one real task from your own repository rather than a sample prompt.

Do I need a GPU to run a coding agent on my own model?

In practice yes. CPU-only inference works and is fine for single questions, but an agent sends many requests per task and each one re-reads a long history, so a slow token rate turns a two-minute task into an hour. Check the PROCESSOR column in ollama ps: any value other than 100% GPU means part of the model is running on the CPU, and the token rate drops sharply.