SSD Nodes Learn Hosting plans →
Guides Matt ConnorBy Matt Connor

Run Nemotron 3.5 Lightning on a VPS

Run NVIDIA Nemotron 3.5 Lightning with Ollama on your own server: the exact tag to pull, how much RAM it needs, and if CPU only is fast enough.

What Nemotron 3.5 Lightning is for

Nemotron 3.5 Lightning is NVIDIA's open 30B mixture-of-experts model, released in August 2026, and it is built for agents that run for hours rather than for one chat window. MoE (mixture of experts) means the weights are split into many expert sub-networks, and each token is routed through only a few of them. NVIDIA's model card gives 30 billion total parameters with 3 billion active per token. You pay for the large number in memory. You get the small number back as speed.

That trade is the reason to look at this model for a server you rent. An agent doing real work sends thousands of short requests across a day, so throughput per dollar decides whether it can live on your own box. A model that takes 40 seconds per reply is a usable assistant and a poor agent, because one task makes twenty calls and you wait for every one of them.

NVIDIA describes the architecture as hybrid: interleaved Mamba-2 and MoE layers with select attention layers. The model card gives a maximum context length up to 1M tokens and an OpenMDW-1.1 license, marked ready for commercial use. Primary languages are English and code, with Spanish, French, German, Italian and Japanese also listed.

Artificial Analysis published launch measurements in August 2026 showing nearly 670 output tokens per second, on a pre-release DeepInfra endpoint serving the NVFP4 weights. That is a hosted GPU endpoint. Read it as what the architecture allows, not as what your VPS will do.

Which Ollama tag fits which VPS

The Ollama library publishes several builds of the same weights. What changes between them is quantisation, which is how many bits each weight is stored in, and that changes the download size a lot.

ChartDownload size by Ollama tag, GB (Ollama library, checked November 2026)
The data behind this chart
[
  {
    "label": "30b-a3b-q4_K_M",
    "size_gb": 25
  },
  {
    "label": "30b-a3b-q8_0",
    "size_gb": 35
  },
  {
    "label": "30b-a3b-bf16",
    "size_gb": 66
  },
  {
    "label": "30b-a3b-mlx",
    "size_gb": 19
  }
]

The tags named latest, 30b and 30b-a3b now carry a digest of their own, listed at 19 to 25 GB since the library updated them in mid-October 2026. Queried directly from the registry on November 3, 2026, that pointer returned exactly the same files as 30b-a3b-q4_K_M, so the default download on a VPS is the 25 GB four-bit build with the full 1M context. The 19 GB lower end of the listed range matches the MLX builds, but automatic selection in Ollama v0.40.14 depends on the operating system and architecture: it prefers MLX on macOS arm64 and llama.cpp, then GGML, then MLX elsewhere, so an MLX-capable Linux NVIDIA server still prefers the llama.cpp variant. Q8_0 is 35 GB and bf16 is 66 GB, both also at 1M. The MLX builds at 19 GB cap at 256K context and run on Ollama's MLX engine rather than on llama.cpp. That engine is not limited to Apple silicon: Ollama v0.40.14 ships a separate Linux bundle, ollama-linux-amd64-mlx.tar.zst (about 1.15 GB), built for CUDA 13, and Ollama's development docs list CUDA 13 or newer and cuDNN 9 or newer for MLX on Linux. The install script does not add that bundle, and without it a pull of an MLX tag stops with this model requires MLX support, but the MLX runtime is not available. Because that CUDA build is the only Linux MLX runtime Ollama publishes, the MLX tags are an option only on a VPS with an NVIDIA GPU, a driver new enough for CUDA 13 and that bundle installed. On a CPU-only plan they are the wrong choice. The library also lists 30b-mlx and 30b-a3b-nvfp4, which both have the same digest as 30b-a3b-mlx even though the second format name sounds like NVIDIA's, plus 30b-a3b-mxfp8 at 34 GB and 30b-a3b-mlx-bf16 at 66 GB. All of them also cap at 256K, so none of them gives you the full window.

Those are download sizes, not a memory requirement. NVIDIA does not publish a minimum VRAM (video RAM) figure for the Ollama builds, so treat the download size as a floor and nothing more. The weights have to be resident somewhere, in GPU memory if the card holds them and in system RAM otherwise, and the KV cache (key/value cache, the model's per-token memory of the conversation) is added on top. The real number for your hardware comes from a command, not from arithmetic, and it is below. If you have not settled on a quantisation level yet, what Q4, Q8 and FP16 each cost you covers what gets given up at each step.

Pull the exact tag, never latest

latest is a moving pointer. When the library republishes it, your agent's behaviour changes on the next pull with nothing in your notes to explain why. That has already happened once with this model: in mid-October 2026 the library moved latest to a new digest. Name the tag.

curl -fsSL https://ollama.com/install.sh | sh
ollama --version
ollama pull nemotron-3.5-lightning:30b-a3b-q4_K_M

The install script sets up a systemd service that runs as the ollama user and keeps models under /usr/share/ollama/.ollama/models. That path is on the root filesystem on most VPS images, so check for room before you ask for 25 GB. If that filesystem is tight, where Ollama stores its models and how to move them is worth reading before the pull rather than after it has filled the disk.

df -h /usr/share/ollama

A pull that stops partway and reports no space left on device means exactly that, and the partial blobs stay on disk until you delete them. Then confirm what landed:

ollama show nemotron-3.5-lightning:30b-a3b-q4_K_M

ollama show prints the architecture, the parameter count, the context length and the quantisation the file actually carries. If any of those disagree with the library page, you pulled a different tag than you meant to.

Serve it, and check where it actually ran

sudo systemctl enable --now ollama
ollama run nemotron-3.5-lightning:30b-a3b-q4_K_M "Reply with one word: ready"

While the model is still loaded, in a second shell:

ollama ps

This is the command that answers the memory question for your machine. ollama ps prints the loaded model, the size it occupies in memory, and a PROCESSOR column. 100% GPU means all of it is in VRAM. 100% CPU means none of it is, and every token is computed by the processor out of system RAM. A split such as 65%/35% CPU/GPU means the layers did not all fit, and the CPU share sets your speed. Do not estimate the requirement. Load it and read this line.

If it cannot load at all, Ollama refuses the load and returns an error to the client rather than crashing. The service log names the cause, so read it with journalctl -u ollama -n 50 and look for this line:

model is too large for system memory

Some Ollama releases also print both sizes in the client error, as in model requires more system memory (28.4 GiB) than is available (15.6 GiB). The exact wording of that message depends on your version, so the log line is the reliable check.

Is a CPU-only VPS fast enough?

A general purpose VPS has no GPU, so the CPU does all the work and reads every weight it needs out of system RAM. MoE helps here, because only about 3 billion of the 30 billion parameters are touched per token, so the arithmetic per token is far smaller than for a dense 30B model. Memory is not helped at all. All 30 billion parameters must stay resident, because the router can pick any expert for any token.

So CPU-only inference on this model is limited by memory bandwidth rather than by core count. Adding vCPUs to a plan that already has a reasonable number of them changes very little. What you need is enough RAM to hold the weights plus your KV cache, and the fastest memory the plan gives you.

Measure it before you commit an agent to it, using the method in measuring tokens per second for a local LLM:

ollama run --verbose nemotron-3.5-lightning:30b-a3b-q4_K_M "Write a 200 word summary of TCP slow start."

The eval rate line printed at the end is your generation speed in tokens per second. That single number decides the question, because an agent's wall-clock time is dominated by it. Divide the length of reply you expect in tokens by that rate, and if the answer in seconds is longer than you are willing to wait, capping the output with num_predict is the one lever that bounds a single call without changing the hardware.

ChartAverage seconds per Intelligence Index task (Artificial Analysis, published August 2026)
The data behind this chart
[
  {
    "label": "Nemotron 3.5 Lightning",
    "sec_per_task": 30
  },
  {
    "label": "gpt-oss-120b",
    "sec_per_task": 204
  },
  {
    "label": "Qwen3.6 35B",
    "sec_per_task": 210
  }
]

Those are published third-party figures, converted from the per-task minutes Artificial Analysis reported at launch, and they were measured on hosted GPU endpoints rather than on a VPS. Nemotron 3.5 Lightning averaged about 30 seconds per task, where gpt-oss-120b took roughly 204 and Qwen3.6 35B roughly 210. Use them for the shape of the gap, not as a promise about your hardware.

The honest guidance splits on who is waiting. If a person is waiting on the agent, or the agent makes long chains of calls back to back, rent GPU capacity. If it runs on a schedule overnight and nobody is watching, a large-RAM CPU plan is a reasonable home. Either way the setup is the same, and running Ollama on a VPS covers the plan sizing and how a GPU instance compares with paying an API provider per token. The break-even is a utilisation question: a GPU instance bills every hour it exists, while API tokens bill only when used, so an agent that is busy most of the day favours the box you own and an agent that fires twice an hour usually does not. The Ollama software adds nothing to either side of that sum, because it is free to run on a server you rent, and what Ollama charges for and what it does not explains where Ollama Cloud fits if you would rather not rent the hardware at all.

The 1M context window is not free

1M tokens is the model's maximum, and Ollama does not hand it to you by default. Ollama serves a much smaller default window and drops the oldest tokens once a conversation passes it. Nothing is logged at the default log level when that happens, so for an agent it looks like the model forgetting the beginning of its own task.

Set the window on purpose. For the whole server, edit the service:

sudo systemctl edit ollama

Add this, then run sudo systemctl restart ollama:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"

Per request, send num_ctx in the options object instead:

curl http://localhost:11434/api/chat -d '{
  "model": "nemotron-3.5-lightning:30b-a3b-q4_K_M",
  "messages": [{"role": "user", "content": "Say ready"}],
  "options": {"num_ctx": 32768},
  "stream": false
}'

Every increase costs memory, because the KV cache grows with the number of tokens you allow. Raise the value, restart, then run ollama ps again and watch the reported size climb. If the PROCESSOR column flips from 100% GPU to a split after that change, the KV cache pushed model layers out of VRAM and your speed will fall hard. Choosing num_ctx in Ollama works through that trade in detail. Do not set 1000000 just because the model card allows it, because the allocation happens up front and the load fails if it does not fit.

Wiring it into an always-on agent

Ollama's library page for this model documents a shortcut that starts a supported agent already pointed at it:

ollama launch claude --model nemotron-3.5-lightning

The page lists claude, opencode, openclaw and hermes in that position. The subcommand needs a current Ollama, so check ollama --version first, and if it is missing, point the agent at the API yourself. Ollama exposes an OpenAI-compatible endpoint, which most agent harnesses accept:

export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=ollama

Ollama ignores the key, but most clients refuse to start without one set. The harness side of this is covered in pointing a coding agent at Ollama and in building your own OpenClaw agent.

Two server settings matter once the agent runs unattended. OLLAMA_KEEP_ALIVE controls how long a model stays in memory after the last request, and the default unloads it after five minutes, so the next call pays the full load time again. On a 25 GB file without a GPU that pause is long enough to break a timeout. Set OLLAMA_KEEP_ALIVE=-1 to keep it resident. OLLAMA_HOST=0.0.0.0:11434 makes the API reachable from other machines, and the API is unauthenticated by default, so open it only behind a firewall rule or a private network.

Failure modes, with the strings you will see

The pull fails immediately. Error: pull model manifest: file does not exist means that tag does not exist. Tag names are exact strings, so copy one from the library page rather than guessing at a quantisation suffix.

The model will not load. The request fails, and journalctl -u ollama shows model is too large for system memory. That means the tag is too big for this plan as configured. Drop to a smaller quantisation, or lower OLLAMA_CONTEXT_LENGTH, because the KV cache is counted inside that requirement.

Nothing answers on port 11434. curl: (7) Failed to connect to localhost port 11434 means the service is not running, or not listening where you expect. Read systemctl status ollama and journalctl -u ollama -n 50. If you also started ollama serve by hand, the second copy exits with Error: listen tcp 127.0.0.1:11434: bind: address already in use.

It answers, very slowly. Check ollama ps before changing anything. Any CPU share in the PROCESSOR column on a GPU machine means part of the model spilled out of VRAM, so lower the context or take the smaller quantisation. On a machine with no GPU, slow is the expected outcome and no setting repairs it.

The agent forgets its instructions partway through a task. The conversation passed the context window and the oldest tokens were discarded silently. Raise OLLAMA_CONTEXT_LENGTH, confirm with ollama ps that the model still fits, and if it no longer does, the fix is a larger machine rather than a smaller window.

Where this model sits against the alternatives

A 30B MoE is a large thing to host for a small job. If a dense 8B model already handles your task, it will cost far less to run and load in seconds, and Qwen 3 at 8B and 27B on a VPS is the direct comparison for that decision. For a wider survey of what a given plan can actually hold, start from which AI models you can self-host. If you plan to serve several agents at once rather than one, read Ollama compared with vLLM first. Ollama can process requests to one model in parallel, but OLLAMA_NUM_PARALLEL ships at 1, so out of the box a second agent's request waits in a queue until the first finishes. For this model, raising it does not help either: as of Ollama v0.40.14 the scheduler forces the nemotron_h_moe architecture this tag carries back to one request at a time and logs model architecture does not currently support parallel requests. On models that do accept it, raising it costs memory: Ollama's FAQ says the RAM required scales by OLLAMA_NUM_PARALLEL times OLLAMA_CONTEXT_LENGTH, so on a large model that multiplication is where a single-user setup stops scaling.

FAQ

Which Nemotron 3.5 Lightning tag should I pull on a Linux VPS?

Use nemotron-3.5-lightning:30b-a3b-q4_K_M. It is 25 GB, it carries the full 1M maximum context, and it is the build the latest, 30b and 30b-a3b pointers returned when queried on November 3, 2026, even though the library moved those pointers to a separate digest in mid-October. Name it explicitly instead of pulling latest, so a future republish of that pointer cannot change your agent's behaviour without you noticing. The mlx tags, and 30b-a3b-nvfp4, which shares the 30b-a3b-mlx digest, cap at 256K and need Ollama's MLX runtime. On Linux that runtime is a separate CUDA 13 bundle the install script does not add, so these tags only make sense on an NVIDIA GPU server with that bundle installed, and they will not help you on a CPU-only VPS.

How much RAM does Nemotron 3.5 Lightning need?

NVIDIA does not publish a minimum memory figure for the Ollama builds, so measure rather than estimate. Pull the tag, run the model once, and read ollama ps while it is loaded: it prints the size actually occupied and whether it landed on the GPU or the CPU. The download size, 25 GB for the default tag, is a floor, because the KV cache is added on top and grows with the context window you set. If the plan is too small, Ollama refuses the load and its service log records model is too large for system memory.

Can I run Nemotron 3.5 Lightning on a VPS with no GPU?

Yes, if the plan has enough RAM to hold the weights, and the MoE design helps because only about 3 of the 30 billion parameters are computed per token. Speed is the catch. Without a GPU the model is limited by memory bandwidth, so adding vCPUs barely moves the result. Run ollama run --verbose with a fixed prompt, read the eval rate line, and judge that number against your agent's deadline. For a batch job overnight it is often fine. For anything a person waits on, it usually is not.

Why does Ollama not give me the full 1M context window?

1M is the model's maximum, not Ollama's default. Ollama applies a much smaller window and discards the oldest tokens once a conversation exceeds it, with no error printed, which reads as the agent forgetting its own instructions. Set OLLAMA_CONTEXT_LENGTH on the systemd service, or pass num_ctx per request. Raise it in steps and re-check ollama ps each time, because the KV cache memory scales with the window and can push model layers off the GPU.

Is Nemotron 3.5 Lightning free to use commercially?

NVIDIA's model card places the model under the OpenMDW-1.1 license and marks it ready for commercial use, as checked on November 3, 2026. The license file Ollama ships with its download is the NVIDIA Open Model License Agreement, so read both before you rely on either. That covers the weights you download and run yourself. It says nothing about the other software in your stack, so check the licences of the agent harness and any tools you connect to it separately, and read the current model card before you rely on this for anything contractual.

Was this article useful?