Ollama Cloud vs your own server: what changes

What Ollama Cloud changes, and what it does not

Ollama Cloud runs the model on ollama.com instead of on your own hardware, while keeping the same ollama command and the same REST API you already call. Two things change: the model name you ask for, and where the credential lives. The rest of your application stays exactly as it is.

That convenience is also the risk. A request to a cloud model looks the same in your code as a request to a local one, so it is easy to lose track of which prompts run on a machine you control and which are sent to a company you do not. This guide draws that line, then shows how to keep a local model as a fallback so a single configuration value decides which side of it you are on.

If you have not stood up the local side yet, start with running Ollama on your own VPS first. Everything below assumes a working ollama on a Linux box.

Two ways to reach Ollama Cloud

There are two paths to the hosted models, and they are not interchangeable. Which one you take decides where the credential is stored, what model name you write, and what a packet capture on your server would show.

Path one: your local daemon forwards the request. You sign in once, then ask for a model whose name ends in -cloud.

ollama signin
ollama pull gpt-oss:120b-cloud
ollama run gpt-oss:120b-cloud

ollama signin links this machine to your ollama.com account. ollama signout unlinks it. After signing in, your application keeps talking to the local port it always used:

curl http://localhost:11434/api/chat -d '{
  "model": "gpt-oss:120b-cloud",
  "messages": [{"role": "user", "content": "Why is the sky blue?"}],
  "stream": false
}'

Read that URL again. It says localhost, and the inference is not happening there. The local daemon recognises the -cloud suffix, forwards the request to ollama.com, and streams the answer back to you. This is the whole point of path one: an application already pointed at the Ollama API on port 11434 needs no code change whatsoever, only a different model string.

Path two: your client calls ollama.com directly. Here the local daemon is not involved at all. Create a key at https://ollama.com/settings/keys, then send it as a bearer token.

export OLLAMA_API_KEY=your_api_key
curl https://ollama.com/api/chat \
  -H "Authorization: Bearer $OLLAMA_API_KEY" \
  -d '{
    "model": "gpt-oss:120b",
    "messages": [{"role": "user", "content": "Why is the sky blue?"}],
    "stream": false
  }'

Look at the model name. On path two it is gpt-oss:120b, with no -cloud suffix. The suffix exists to tell your local daemon to forward the request upstream, so it belongs on path one only. When you call https://ollama.com you are already there, and you write the plain name. The authoritative list of those names comes from the host itself:

curl https://ollama.com/api/tags

Run that instead of trusting any model list printed in an article, including this one. The catalogue changes, and api/tags is always current.

Which client calls change, and which do not

The official Python and JavaScript libraries take the host and the headers when you construct the client. Nothing after that line differs. On path one, the constructor is empty, because the default is the local daemon:

from ollama import Client

client = Client()

messages = [{'role': 'user', 'content': 'Why is the sky blue?'}]

for part in client.chat('gpt-oss:120b-cloud', messages=messages, stream=True):
  print(part['message']['content'], end='', flush=True)

On path two, the constructor carries the host and the token:

import os
from ollama import Client

client = Client(
    host="https://ollama.com",
    headers={'Authorization': 'Bearer ' + os.environ.get('OLLAMA_API_KEY')}
)

messages = [{'role': 'user', 'content': 'Why is the sky blue?'}]

for part in client.chat('gpt-oss:120b', messages=messages, stream=True):
  print(part['message']['content'], end='', flush=True)

The client.chat() call, the streaming loop, the message list and the response shape are identical in both. That is why moving between hosted and self-hosted is a configuration change and not a rewrite. The OpenAI-compatible surface behaves the same way locally: point an OpenAI SDK at http://localhost:11434/v1/ with api_key='ollama', which the local server requires and then ignores.

Where the credential lives, and who can spend it

On path two the credential is OLLAMA_API_KEY in your environment. Keep it out of shell history and out of your repository. On a systemd service, put it in an Environment= line or an environment file owned by root with mode 600.

Path one is the one that surprises people. The sign-in belongs to the daemon, not to you. The Ollama FAQ documents the service identity on Linux at /usr/share/ollama/.ollama/id_ed25519.pub, owned by the ollama service user. There is no per-request authentication on the local API, so every caller that can reach port 11434 inherits your account and spends your quota. That is fine while the daemon listens on loopback. The moment you set OLLAMA_HOST=0.0.0.0:11434 to reach it from another machine, an open port becomes an open billing relationship, which is why you should read how to put authentication in front of an Ollama endpoint before you widen the bind address.

Why the same model gives you less context locally

This is the difference that catches people who assume a model behaves identically on both paths. It does not, and the cause is memory.

Ollama picks a local default context length from the video memory it finds on the box.

[
  {
    "label": "Under 24 GiB VRAM",
    "default_context_tokens": "4,096"
  },
  {
    "label": "24 to 48 GiB VRAM",
    "default_context_tokens": "32,768"
  },
  {
    "label": "48 GiB VRAM or more",
    "default_context_tokens": "262,144"
  }
]

A VPS with no GPU sits in the bottom tier, so a local model starts at 4,096 tokens of context, while a machine with a large card starts at 262,144. Cloud models ignore these tiers: Ollama documents them as set to their maximum context length by default, because the memory holding that context is not yours.

So the same prompt that works against gpt-oss:120b-cloud can be silently truncated against a local model on a small box. Raise the local ceiling explicitly:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Under systemd, set it as Environment="OLLAMA_CONTEXT_LENGTH=32768" with systemctl edit ollama.service, then systemctl daemon-reload && systemctl restart ollama. Be aware of what you are buying: a longer context means a larger key-value cache, and that cache is RAM the model needs on top of its weights. Push it too far and generation slows down or the model fails to load. Setting num_ctx and OLLAMA_CONTEXT_LENGTH correctly covers the arithmetic, and which models fit the memory you actually have covers the weights side.

What actually leaves your machine

Be exact about this, because it is the reason most readers self-host in the first place.

Running locally, nothing leaves. Ollama's privacy policy states plainly that for local use, "We do not collect, store, transmit, or have access to your prompts, responses, model interactions, or other content you process locally." One caveat: pulling a model is still a download from ollama.com, and the policy lists "model download metadata" and your IP address among the things collected. The registry learns which models you fetched. It does not learn what you asked them.

Running on the cloud path, the full prompt and the full completion go to a third party. There is no partial version of this. Every token you send and every token you receive is processed on ollama.com. The policy says the company processes "your prompts and responses transiently to provide the service" and does "not use your inputs or outputs to train any AI models," and describes "technical measures designed to minimize retention of prompt and response content." That is a reasonable commitment. It is still a commitment by someone else about data you handed over, rather than a property of your own machine. Judge it the way you would judge any vendor promise, and re-read it before you send anything you are contractually or legally obliged to keep on your own infrastructure.

The trap is path one. Your code says http://localhost:11434, your firewall rules are unchanged, and your prompt still crosses the internet, because the -cloud suffix on the model name is doing the routing. A localhost URL tells you nothing about where inference happened. The model name does.

Keeping a local model as the fallback

Because both paths speak the same API, you can make the choice a runtime setting instead of a fork in your code.

The simplest version needs no code at all. Keep your application pointed at the local daemon and put the model name in configuration. Set it to llama3.2 and you run on your own box. Set it to gpt-oss:120b-cloud and the same daemon forwards to ollama.com. One environment variable, no redeploy.

When you want the local model to be the default and the cloud to catch the overflow, construct both clients and choose per request:

import os
from httpx import ConnectError
from ollama import Client, ResponseError

LOCAL_MODEL = os.environ.get("LOCAL_MODEL", "llama3.2")
CLOUD_MODEL = os.environ.get("CLOUD_MODEL", "gpt-oss:120b")

local = Client(host="http://127.0.0.1:11434")
cloud = Client(
    host="https://ollama.com",
    headers={"Authorization": "Bearer " + os.environ["OLLAMA_API_KEY"]},
)

def chat(messages):
    try:
        return local.chat(LOCAL_MODEL, messages=messages)
    except (ConnectError, ResponseError) as err:
        print(f"local inference failed ({err}); sending this prompt to ollama.com")
        return cloud.chat(CLOUD_MODEL, messages=messages)

httpx arrives as a dependency of the ollama package, so there is nothing extra to install. ConnectError covers a daemon that is down. ResponseError covers a daemon that is up and refusing, for example when the local model was never pulled.

That print line is not decoration. A silent fallback means a prompt you intended to keep on your own hardware quietly goes to a third party the first time your daemon restarts during an upgrade. Log every fallback, and on anything sensitive, raise the error instead of falling back. The safest fallback policy for a privacy-driven deployment is to fail loudly.

One more thing makes the local side a credible default: keep the model resident. A cold load on a CPU-only VPS can take tens of seconds, which is what pushes people to the cloud path in the first place. Holding the model in memory with keep_alive removes that first-request penalty.

What to compare before you commit

Do not compare on price alone, and do not trust a price you read in an article, including the date on this one. Compare four things, and check each on the vendor's own pages:

• Model availability. Run curl https://ollama.com/api/tags for the current hosted catalogue. The models you can run locally are limited by your RAM and VRAM instead.
• Context limits. Hosted models default to their maximum. Local models default by VRAM tier, as above. If your workload is long documents, this decides the question by itself.
• Rate limits. Hosted inference is metered. Exceed the limit and the API answers 429 Too Many Requests. Your own server has no rate limit and a hard concurrency ceiling instead, which is a different failure and often a worse one.
• Retention policy. Read the actual policy text, note the date you read it, and re-check it before any renewal.

For the money side, do not redo the arithmetic here. Where a GPU VPS overtakes per-token billing works through the break-even properly, including the part people forget: an idle GPU server bills the same as a busy one.

What about a router in front of several providers?

The third option is a router, a proxy that speaks one API to your application and fans requests out to several backends. A self-hosted LiteLLM proxy or a hosted service such as OpenRouter both do this. The appeal is real: one client configuration, several models, and failover when a provider has a bad hour. It is the natural extension of the fallback pattern above, generalised past two backends. Be clear about the cost, though. A hosted router is another operator that sees your prompts, so the retention question you asked of one vendor now has to be asked of two. A self-hosted router keeps that hop on your own machine and hands you a second service to run, patch and monitor. Routers solve model choice and availability. They do not solve privacy, because the prompt still ends up wherever the route sends it.

The errors you will actually see

The API documents the status codes, and each one points at a different problem. 429 Too Many Requests means you hit a rate limit, so back off and retry rather than reconnecting in a loop. 502 Bad Gateway is the one specific to this topic: it is returned when a cloud model cannot be reached, so on path one it means your daemon is fine and the upstream is not. 404 Not Found on a model name usually means the suffix does not match the host, a -cloud name sent straight to https://ollama.com, or a bare name sent to a daemon you never signed in on. Errors arrive as JSON, and mid-stream they appear as a line such as {"error":"an error was encountered while running the model"} inside the NDJSON response, which is why a naive streaming client can print a partial answer and then stop with no explanation. Parse each streamed line and check for an error key.

On the local side the classic is a connection refused on port 11434, which means the daemon is not running: check systemctl status ollama. The other classic is a request that works from your shell and fails from a container, because the container's localhost is not the host's.

And the failure mode with no error message at all: no internet. The cloud path stops working completely, while the local path does not notice. If your machine goes on the road or your provider has a routing incident, that difference is the entire product.

When each choice is right

Use Ollama Cloud when the work is bursty, when the model is too large for your VPS, or when you are still deciding whether a model is worth building on. Paying per request beats paying for an idle GPU that runs for twenty minutes a day, and a 120-billion-parameter model is not going to fit on a box you rent for the price of a lunch.

Run it yourself when the prompts must not leave your infrastructure, when the machine has to work offline, or when load is steady enough that a rented GPU stays busy. Steady load is the honest signal: metered inference is expensive precisely when it is running all the time. If you get there and Ollama's single-request throughput becomes the bottleneck, vLLM handles concurrent load better than Ollama does, and that is a change of engine rather than a change of host.

Most real deployments end up using both, which is fine as long as the split is deliberate. Put the model name in configuration, log every fallback, and you will always be able to answer the only question that matters here: which of these prompts left the building?

FAQ

Does Ollama Cloud see my prompts?

Yes. On the hosted path the full prompt and the full completion are sent to ollama.com and processed there. Ollama's privacy policy says it processes "your prompts and responses transiently to provide the service" and does "not use your inputs or outputs to train any AI models," and describes measures designed to minimise retention. That is a vendor commitment about data you have already handed over. For local models the same policy says the company does "not collect, store, transmit, or have access to your prompts, responses, model interactions, or other content you process locally." If the requirement is that content never leaves your infrastructure, only the local path satisfies it.

Why does my app still point at localhost when the model runs in the cloud?

Because the local daemon is acting as a proxy. When you run ollama signin and then request a model whose name ends in -cloud, your daemon forwards that request to ollama.com and streams the answer back through port 11434. Your application URL does not change, which is the point: no code edit is needed. It also means the localhost address tells you nothing about where inference happened. Check the model name, not the URL. A -cloud suffix means the prompt crossed the internet.

Why does the same model give me a much shorter context locally?

Ollama chooses a local default from available video memory: about 4k tokens below 24 GiB of VRAM, 32k between 24 and 48 GiB, and 256k at 48 GiB and above. A VPS with no GPU falls in the bottom tier. Cloud models are set to their maximum context length by default, because the memory holding it belongs to the provider. Raise the local value with OLLAMA_CONTEXT_LENGTH, either as OLLAMA_CONTEXT_LENGTH=32768 ollama serve or as an Environment= line under systemctl edit ollama.service. Remember that a longer context needs a bigger key-value cache in RAM, so raising it on a small server can slow generation or stop the model loading.

Can I fall back to a local model automatically when the cloud is unreachable?

Yes, and it is a few lines, because both paths speak the same API. Build two Client objects, one with no host argument for the local daemon and one with host="https://ollama.com" plus an Authorization: Bearer header, then catch httpx.ConnectError and ollama.ResponseError around the first call. Decide the direction deliberately. Local first with a cloud fallback means a prompt you meant to keep private can leave the machine during a routine daemon restart, so log every fallback, and for sensitive workloads raise the error instead of falling back.