Run Nemotron 3.5 Lightning on a VPS

What Nemotron 3.5 Lightning is for

Nemotron 3.5 Lightning is NVIDIA's open 30B mixture-of-experts model, released in August 2026, and it is built for agents that run for hours rather than for one chat window. MoE (mixture of experts) means the weights are split into many expert sub-networks, and each token is routed through only a few of them. NVIDIA's model card gives 30 billion total parameters with 3 billion active per token. You pay for the large number in memory. You get the small number back as speed.

That trade is the reason to look at this model for a server you rent. An agent doing real work sends thousands of short requests across a day, so throughput per dollar decides whether it can live on your own box. A model that takes 40 seconds per reply is a usable assistant and a poor agent, because one task makes twenty calls and you wait for every one of them.

NVIDIA describes the architecture as hybrid: interleaved Mamba-2 and MoE layers with select attention layers. The model card gives a maximum context length up to 1M tokens and an OpenMDW-1.1 license, marked ready for commercial use. Primary languages are English and code, with Spanish, French, German, Italian and Japanese also listed.

Artificial Analysis published launch measurements in August 2026 showing nearly 670 output tokens per second, on a pre-release DeepInfra endpoint serving the NVFP4 weights. That is a hosted GPU endpoint. Read it as what the architecture allows, not as what your VPS will do.

Which Ollama tag fits which VPS

The Ollama library publishes several builds of the same weights. What changes between them is quantisation, which is how many bits each weight is stored in, and that changes the download size a lot.

[
  {
    "label": "30b-a3b-q4_K_M",
    "size_gb": 25
  },
  {
    "label": "30b-a3b-q8_0",
    "size_gb": 35
  },
  {
    "label": "30b-a3b-bf16",
    "size_gb": 66
  },
  {
    "label": "30b-a3b-mlx",
    "size_gb": 23
  }
]

The tags named latest, 30b and 30b-a3b all resolve to the same digest as 30b-a3b-q4_K_M, so the default download is the 25 GB four-bit build with the full 1M context. Q8_0 is 35 GB and bf16 is 66 GB, both also at 1M. The MLX builds at 23 GB are for Apple silicon and cap at 256K context, so they are the wrong choice on a Linux VPS.

Those are download sizes, not a memory requirement. NVIDIA does not publish a minimum VRAM (video RAM) figure for the Ollama builds, so treat the download size as a floor and nothing more. The weights have to be resident somewhere, in GPU memory if the card holds them and in system RAM otherwise, and the KV cache (key/value cache, the model's per-token memory of the conversation) is added on top. The real number for your hardware comes from a command, not from arithmetic, and it is below. If you have not settled on a quantisation level yet, what Q4, Q8 and FP16 each cost you covers what gets given up at each step.

Pull the exact tag, never latest

latest is a moving pointer. When the library republishes it, your agent's behaviour changes on the next pull with nothing in your notes to explain why. Name the tag.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
ollama pull nemotron-3.5-lightning:30b-a3b-q4_K_M

The install script sets up a systemd service that runs as the ollama user and keeps models under /usr/share/ollama/.ollama/models. That path is on the root filesystem on most VPS images, so check for room before you ask for 25 GB.

df -h /usr/share/ollama

A pull that stops partway and reports no space left on device means exactly that, and the partial blobs stay on disk until you delete them. Then confirm what landed:

ollama show nemotron-3.5-lightning:30b-a3b-q4_K_M

ollama show prints the architecture, the parameter count, the context length and the quantisation the file actually carries. If any of those disagree with the library page, you pulled a different tag than you meant to.

Serve it, and check where it actually ran

sudo systemctl enable --now ollama
ollama run nemotron-3.5-lightning:30b-a3b-q4_K_M "Reply with one word: ready"

While the model is still loaded, in a second shell:

ollama ps

This is the command that answers the memory question for your machine. ollama ps prints the loaded model, the size it occupies in memory, and a PROCESSOR column. 100% GPU means all of it is in VRAM. 100% CPU means none of it is, and every token is computed by the processor out of system RAM. A split such as 65%/35% CPU/GPU means the layers did not all fit, and the CPU share sets your speed. Do not estimate the requirement. Load it and read this line.

If it cannot load at all, Ollama refuses cleanly rather than crashing:

Error: model requires more system memory (28.4 GiB) than is available (15.6 GiB)

Is a CPU-only VPS fast enough?

A general purpose VPS has no GPU, so the CPU does all the work and reads every weight it needs out of system RAM. MoE helps here, because only about 3 billion of the 30 billion parameters are touched per token, so the arithmetic per token is far smaller than for a dense 30B model. Memory is not helped at all. All 30 billion parameters must stay resident, because the router can pick any expert for any token.

So CPU-only inference on this model is limited by memory bandwidth rather than by core count. Adding vCPUs to a plan that already has a reasonable number of them changes very little. What you need is enough RAM to hold the weights plus your KV cache, and the fastest memory the plan gives you.

Measure it before you commit an agent to it, using the method in measuring tokens per second for a local LLM:

ollama run --verbose nemotron-3.5-lightning:30b-a3b-q4_K_M "Write a 200 word summary of TCP slow start."

The eval rate line printed at the end is your generation speed in tokens per second. That single number decides the question, because an agent's wall-clock time is dominated by it.

[
  {
    "label": "Nemotron 3.5 Lightning",
    "sec_per_task": 30
  },
  {
    "label": "gpt-oss-120b",
    "sec_per_task": 204
  },
  {
    "label": "Qwen3.6 35B",
    "sec_per_task": 210
  }
]

Those are published third-party figures, converted from the per-task minutes Artificial Analysis reported at launch, and they were measured on hosted GPU endpoints rather than on a VPS. Nemotron 3.5 Lightning averaged about 30 seconds per task, where gpt-oss-120b took roughly 204 and Qwen3.6 35B roughly 210. Use them for the shape of the gap, not as a promise about your hardware.

The honest guidance splits on who is waiting. If a person is waiting on the agent, or the agent makes long chains of calls back to back, rent GPU capacity. If it runs on a schedule overnight and nobody is watching, a large-RAM CPU plan is a reasonable home. Either way the setup is the same, and running Ollama on a VPS covers the plan sizing and how a GPU instance compares with paying an API provider per token. The break-even is a utilisation question: a GPU instance bills every hour it exists, while API tokens bill only when used, so an agent that is busy most of the day favours the box you own and an agent that fires twice an hour usually does not.

The 1M context window is not free

1M tokens is the model's maximum, and Ollama does not hand it to you by default. Ollama serves a much smaller default window and drops the oldest tokens once a conversation passes it. Nothing is logged when that happens, so for an agent it looks like the model forgetting the beginning of its own task.

Set the window on purpose. For the whole server, edit the service:

sudo systemctl edit ollama

Add this, then run sudo systemctl restart ollama:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"

Per request, send num_ctx in the options object instead:

curl http://localhost:11434/api/chat -d '{
  "model": "nemotron-3.5-lightning:30b-a3b-q4_K_M",
  "messages": [{"role": "user", "content": "Say ready"}],
  "options": {"num_ctx": 32768},
  "stream": false
}'

Every increase costs memory, because the KV cache grows with the number of tokens you allow. Raise the value, restart, then run ollama ps again and watch the reported size climb. If the PROCESSOR column flips from 100% GPU to a split after that change, the KV cache pushed model layers out of VRAM and your speed will fall hard. Choosing num_ctx in Ollama works through that trade in detail. Do not set 1000000 just because the model card allows it, because the allocation happens up front and the load simply fails.

Wiring it into an always-on agent

Ollama's launch post for this model documents a shortcut that starts a supported agent already pointed at it:

ollama launch claude --model nemotron-3.5-lightning

The post documents claude, opencode, openclaw and hermes in that position. The subcommand needs a current Ollama, so check ollama --version first, and if it is missing, point the agent at the API yourself. Ollama exposes an OpenAI-compatible endpoint, which most agent harnesses accept:

export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=ollama

Ollama ignores the key, but most clients refuse to start without one set. The harness side of this is covered in pointing a coding agent at Ollama and in building your own OpenClaw agent.

Two server settings matter once the agent runs unattended. OLLAMA_KEEP_ALIVE controls how long a model stays in memory after the last request, and the default unloads it after five minutes, so the next call pays the full load time again. On a 25 GB file without a GPU that pause is long enough to break a timeout. Set OLLAMA_KEEP_ALIVE=-1 to keep it resident. OLLAMA_HOST=0.0.0.0:11434 makes the API reachable from other machines, and it carries no authentication of any kind, so open it only behind a firewall rule or a private network.

Failure modes, with the strings you will see

The pull fails immediately. Error: pull model manifest: file does not exist means that tag does not exist. Tag names are exact strings, so copy one from the library page rather than guessing at a quantisation suffix.

The model will not load. Error: model requires more system memory (28.4 GiB) than is available (15.6 GiB) means the tag is too big for this plan as configured. Drop to a smaller quantisation, or lower OLLAMA_CONTEXT_LENGTH, because the KV cache is counted inside that requirement.

Nothing answers on port 11434. curl: (7) Failed to connect to localhost port 11434 means the service is not running, or not listening where you expect. Read systemctl status ollama and journalctl -u ollama -n 50. If you also started ollama serve by hand, the second copy exits with Error: listen tcp 127.0.0.1:11434: bind: address already in use.

It answers, very slowly. Check ollama ps before changing anything. Any CPU share in the PROCESSOR column on a GPU machine means part of the model spilled out of VRAM, so lower the context or take the smaller quantisation. On a machine with no GPU, slow is the expected outcome and no setting repairs it.

The agent forgets its instructions partway through a task. The conversation passed the context window and the oldest tokens were discarded silently. Raise OLLAMA_CONTEXT_LENGTH, confirm with ollama ps that the model still fits, and if it no longer does, the fix is a larger machine rather than a smaller window.

Where this model sits against the alternatives

A 30B MoE is a large thing to host for a small job. If a dense 8B model already handles your task, it will cost far less to run and load in seconds, and Qwen 3 at 8B and 27B on a VPS is the direct comparison for that decision. For a wider survey of what a given plan can actually hold, start from which AI models you can self-host. If you plan to serve several agents at once rather than one, read Ollama compared with vLLM first, because Ollama does not batch concurrent requests the way a production inference server does, and that is where a single-user setup stops scaling.

FAQ

Which Nemotron 3.5 Lightning tag should I pull on a Linux VPS?

Use nemotron-3.5-lightning:30b-a3b-q4_K_M. It is 25 GB, it carries the full 1M maximum context, and it is the same digest the latest, 30b and 30b-a3b tags point at as of August 2026. Name it explicitly instead of pulling latest, so a future republish of that pointer cannot change your agent's behaviour without you noticing. The mlx tags are Apple silicon builds and will not help you on Linux.

How much RAM does Nemotron 3.5 Lightning need?

NVIDIA does not publish a minimum memory figure for the Ollama builds, so measure rather than estimate. Pull the tag, run the model once, and read ollama ps while it is loaded: it prints the size actually occupied and whether it landed on the GPU or the CPU. The download size, 25 GB for the default tag, is a floor, because the KV cache is added on top and grows with the context window you set. If the plan is too small, Ollama refuses with model requires more system memory and names both numbers.

Can I run Nemotron 3.5 Lightning on a VPS with no GPU?

Yes, if the plan has enough RAM to hold the weights, and the MoE design helps because only about 3 of the 30 billion parameters are computed per token. Speed is the catch. Without a GPU the model is limited by memory bandwidth, so adding vCPUs barely moves the result. Run ollama run --verbose with a fixed prompt, read the eval rate line, and judge that number against your agent's deadline. For a batch job overnight it is often fine. For anything a person waits on, it usually is not.

Why does Ollama not give me the full 1M context window?

1M is the model's maximum, not Ollama's default. Ollama applies a much smaller window and discards the oldest tokens once a conversation exceeds it, with no error printed, which reads as the agent forgetting its own instructions. Set OLLAMA_CONTEXT_LENGTH on the systemd service, or pass num_ctx per request. Raise it in steps and re-check ollama ps each time, because the KV cache memory scales with the window and can push model layers off the GPU.

Is Nemotron 3.5 Lightning free to use commercially?

NVIDIA's model card places the model under the OpenMDW-1.1 license and marks it ready for commercial use. That covers the weights you download and run yourself. It says nothing about the other software in your stack, so check the licences of the agent harness and any tools you connect to it separately, and read the current model card before you rely on this for anything contractual.