How to Run Nemotron 3.5 Lightning on VPS
Run Nemotron 3.5 Lightning with Ollama for your server. See the exact tag to pull, RAM wey the 30B MoE model need, and whether CPU-only go fast enough.
Wetin Nemotron 3.5 Lightning dey do
Nemotron 3.5 Lightning na NVIDIA open 30B mixture-of-experts model, wey dem release for August 2026, and dem build am for agents wey dey run for hours instead of just one chat window. MoE (mixture of experts) mean say dem divide the weights into plenty expert sub-networks, and dem dey route each token through only small portion of dem. NVIDIA model card talk say e get 30 billion total parameters with 3 billion wey dey active per token. You dey pay for the big number for memory. You dey get the small number back as speed.
Dat trade-off na the reason why you suppose look this model for server wey you rent. Agent wey dey do real work dey send thousands of short requests throughout the day, so throughput per dollar na wetin go decide whether e fit live for your own box. Model wey dey take 40 seconds per reply na okay assistant but e no good as agent, because one task fit make twenty calls and you go wait for every one of dem.
NVIDIA describe the architecture as hybrid: e get interleaved Mamba-2 and MoE layers with some attention layers. The model card talk say maximum context length fit reach 1M tokens and e get OpenMDW-1.1 license, wey mean say e ready for commercial use. The main languages na English and code, but dem also list Spanish, French, German, Italian and Japanese.
Artificial Analysis publish launch measurements for August 2026 wey show say e fit reach nearly 670 output tokens per second, for one pre-release DeepInfra endpoint wey dey serve the NVFP4 weights. Dat one na hosted GPU endpoint. Make you read am as wetin the architecture fit do, no be wetin your VPS go fit do.
Wich Ollama tag fit which VPS
Ollama library dey publish different builds of the same weights. Wetin dey change between dem na quantisation, wey mean how many bits dem use store each weight. This one fit change download size well well.
Di data wey dey behind dis chart
[
{
"label": "30b-a3b-q4_K_M",
"size_gb": 25
},
{
"label": "30b-a3b-q8_0",
"size_gb": 35
},
{
"label": "30b-a3b-bf16",
"size_gb": 66
},
{
"label": "30b-a3b-mlx",
"size_gb": 19
}
]The tags wey dem name latest, 30b and 30b-a3b now get their own digest. Library list dem between 19 and 25 GB after e update dem for mid-October 2026. When we query the registry directly on October 25, 2026, that pointer return exactly the same files as 30b-a3b-q4_K_M. So, the default download for VPS na the 25 GB four-bit build with the full 1M context. The 19 GB lower end of the listed range match the MLX builds. This suggest say Apple silicon machines dey get those ones instead. Q8_0 na 35 GB and bf16 na 66 GB. Both still dey use 1M. The MLX builds wey dey 19 GB na for Apple silicon and dem stop at 256K context. So dem no be correct choice for Linux VPS. The library also list 30b-mlx and 30b-a3b-nvfp4. Both get the same digest as 30b-a3b-mlx, even though the second format name sound like NVIDIA own. E also list 30b-a3b-mxfp8 at 34 GB and 30b-a3b-mlx-bf16 at 66 GB. All of dem still stop at 256K. So none of dem give you the full context window.
Those sizes na download sizes, not memory requirement. NVIDIA no publish minimum VRAM (video RAM) figure for the Ollama builds. So treat download size as the minimum floor only. The weights must stay resident somewhere. If the GPU card fit hold dem, dem go dey GPU memory; otherwise dem go dey system RAM. The KV cache (key/value cache, the model memory for each token of the conversation) dey add on top. The real number for your hardware come from a command, not from calculation, and e dey below. If you never choose quantisation level, wetin Q4, Q8 and FP16 each cost you explain wetin you give up for each step.
Pull the exact tag, never latest
latest na moving pointer. When library publish am again, your agent behaviour go change for the next pull, with nothing for your notes to explain why. This don happen once before with this model: for mid-October 2026, library move latest go a new digest. Name the tag.
curl -fsSL https://ollama.com/install.sh | sh
ollama --version
ollama pull nemotron-3.5-lightning:30b-a3b-q4_K_MThe install script set up a systemd service wey dey run as the ollama user and keep models under /usr/share/ollama/.ollama/models. For most VPS images, that path dey for root filesystem. So check whether space dey before you request 25 GB. If that filesystem no get enough space, where Ollama dey store its models and how to move dem worth reading before the pull, instead of after e don fill the disk.
df -h /usr/share/ollamaIf pull stop halfway and report no space left on device, na exactly wetin happen be that. The partial blobs go remain for disk until you delete dem. Then confirm wetin land:
ollama show nemotron-3.5-lightning:30b-a3b-q4_K_Mollama show go print the architecture, parameter count, context length, and quantisation wey the file actually get. If any of dem no match the library page, you pull different tag from the one wey you mean to use.
Serve am, come check where e actually run
sudo systemctl enable --now ollama
ollama run nemotron-3.5-lightning:30b-a3b-q4_K_M "Reply with one word: ready"While the model still dey load, for one second shell:
ollama psDis na the command wey go answer the memory question for your machine. ollama ps go print the model wey load, the size wey e dey take for memory, plus one PROCESSOR column. 100% GPU mean say everything dey inside VRAM. 100% CPU mean say nothing dey inside, and every token na the processor dey compute am from system RAM. One split like 65%/35% CPU/GPU mean say the layers no fit enter everything, and the CPU share na im go determine your speed. No try guess the requirement. Load am make you read dis line.
If e no fit load at all, Ollama go refuse cleanly instead of make e crash:
Error: model requires more system memory (28.4 GiB) than is available (15.6 GiB)CPU-only VPS dey fast enough?
General purpose VPS no get GPU, so CPU go do all the work and read every weight wey e need from system RAM. MoE dey help for here, because na only about 3 billion out of the 30 billion parameters dem dey use for each token. So the arithmetic for each token much smaller than wetin dense 30B model need. Memory no get any benefit at all. All 30 billion parameters must remain resident, because router fit choose any expert for any token.
So CPU-only inference for this model dey limited by memory bandwidth, not by core count. If you add more vCPUs to plan wey already get reasonable number of dem, the difference go small. Wetin you need na enough RAM to hold the weights plus your KV cache, and the fastest memory wey the plan provide.
Measure am before you commit agent to am, using the method for how to measure tokens per second for local LLM:
ollama run --verbose nemotron-3.5-lightning:30b-a3b-q4_K_M "Write a 200 word summary of TCP slow start."The eval rate line wey print for the end na your generation speed in tokens per second. Na this one number decide the question, because agent wall-clock time mostly depend on am. Multiply am by the reply length wey you expect. If the answer longer than the time wey you fit wait, how to cap output with num_predict na the one control wey fit limit one call without changing the hardware.
Di data wey dey behind dis chart
[
{
"label": "Nemotron 3.5 Lightning",
"sec_per_task": 30
},
{
"label": "gpt-oss-120b",
"sec_per_task": 204
},
{
"label": "Qwen3.6 35B",
"sec_per_task": 210
}
]These figures come from published third-party data. Dem convert dem from the per-task minutes wey Artificial Analysis report for launch, and dem measure dem on hosted GPU endpoints, not on VPS. Nemotron 3.5 Lightning average about 30 seconds per task. For this, gpt-oss-120b take roughly 204, while Qwen3.6 35B take roughly 210. Use dem to understand the size of the difference, not as promise for your hardware.
The honest advice depend on who dey wait. If person dey wait for the agent, or the agent dey make long chains of calls one after another, rent GPU capacity. If e dey run on schedule overnight and nobody dey watch am, large-RAM CPU plan fit work well. Either way, the setup na the same, and how to run Ollama on VPS explain plan sizing and how GPU instance compare with paying API provider per token. The break-even na utilisation question: GPU instance bill you for every hour wey e exist, while API tokens bill only when you use dem. So agent wey dey busy most of the day favour the box wey you own, while agent wey fire twice per hour usually no favour am. Ollama software add nothing to either side of that calculation, because e free to run on server wey you rent, and wetin Ollama dey charge for and wetin e no dey charge for explain where Ollama Cloud fit enter if you no want rent the hardware at all.
The 1M context window no be free
1M tokens na the model maximum, and Ollama no dey give you all of am by default. Ollama dey serve much smaller default window and e dey drop the oldest tokens once conversation pass that limit. Nothing dey enter log for default log level when this happen, so for agent e go look like say model forget the beginning of its own task.
Set the window intentionally. For the whole server, edit the service:
sudo systemctl edit ollamaAdd this, then run sudo systemctl restart ollama:
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"For one request, send num_ctx inside the options object instead:
curl http://localhost:11434/api/chat -d '{
"model": "nemotron-3.5-lightning:30b-a3b-q4_K_M",
"messages": [{"role": "user", "content": "Say ready"}],
"options": {"num_ctx": 32768},
"stream": false
}'Every increase dey use more memory, because KV cache dey grow based on the number of tokens wey you allow. Increase the value, restart, then run ollama ps again and watch the reported size dey rise. If the PROCESSOR column change from 100% GPU to a split after that change, KV cache don push model layers comot from VRAM and your speed go drop badly. How to choose num_ctx for Ollama explain this trade-off well. No set 1000000 just because the model card allow am, because allocation happen upfront and the load go simply fail.
Connect am to agent wey dey always on
Ollama library page for this model document shortcut wey start supported agent already pointed to am:
ollama launch claude --model nemotron-3.5-lightningThe page list claude, opencode, openclaw and hermes for that position. This subcommand need current Ollama, so check ollama --version first. If e no dey, point the agent to the API yourself. Ollama expose OpenAI-compatible endpoint wey most agent harnesses accept:
export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=ollamaOllama no use the key, but most clients no go start unless you set one. You fit read about the harness side for pointing coding agent to Ollama and building your own OpenClaw agent.
Two server settings matter when the agent dey run unattended. OLLAMA_KEEP_ALIVE control how long model go stay for memory after the last request. The default go unload am after five minutes, so the next call go pay the full load time again. For a 25 GB file without GPU, that pause fit long enough to break timeout. Set OLLAMA_KEEP_ALIVE=-1 to keep am resident. OLLAMA_HOST=0.0.0.0:11434 make the API reachable from other machines, and e no get any authentication, so open am only behind firewall rule or private network.
Failure modes, with the strings wey you go see
The pull fail immediately. Error: pull model manifest: file does not exist mean say that tag no dey exist. Tag names na exact strings, so copy one from the library page instead of make you dey guess the quantisation suffix.
The model no go load. Error: model requires more system memory (28.4 GiB) than is available (15.6 GiB) mean say the tag too big for this plan as you configure am. Drop go smaller quantisation, or lower OLLAMA_CONTEXT_LENGTH, because the KV cache dey count inside that requirement.
Nothing answer for port 11434. curl: (7) Failed to connect to localhost port 11434 mean say the service no dey run, or e no dey listen where you expect. Read systemctl status ollama and journalctl -u ollama -n 50. If you self start ollama serve by hand, the second copy go exit with Error: listen tcp 127.0.0.1:11434: bind: address already in use.
E answer, but e dey very slow. Check ollama ps before you change anything. Any CPU share for the PROCESSOR column for a GPU machine mean say part of the model don spill comot from VRAM, so lower the context or take the smaller quantisation. For machine wey no get GPU, slow na the expected outcome and no setting fit repair am.
The agent dey forget im instructions as e dey do task. The conversation don pass the context window and dem don silently throway the oldest tokens. Raise OLLAMA_CONTEXT_LENGTH, confirm with ollama ps say the model still fit, and if e no fit again, the fix na bigger machine instead of smaller window.
Wetin this model be compared with the other options
30B MoE na big thing to host for small work. If dense 8B model don already handle your task, e go cost much less to run and e go load within seconds. Read Qwen 3 at 8B and 27B on a VPS for direct comparison before you decide. If you want wider survey of wetin one plan fit actually hold, start with which AI models you can self-host. If you plan serve several agents at once instead of one, read Ollama compared with vLLM first. Ollama fit process requests to one model in parallel, but OLLAMA_NUM_PARALLEL dey ship with 1. So, without extra configuration, request from second agent go wait for queue until the first one finish. For this model, increasing am no go help too. As of Ollama v0.40.11, the scheduler dey force the nemotron_h_moe architecture wey this tag carry back to one request at a time, and e dey log model architecture does not currently support parallel requests. For models wey accept am, increasing am dey use more memory. Ollama FAQ talk say the RAM wey dem need dey scale by OLLAMA_NUM_PARALLEL times OLLAMA_CONTEXT_LENGTH. For large model, na this multiplication dey stop single-user setup from scaling.
FAQ
Which Nemotron 3.5 Lightning tag should I pull on a Linux VPS?
Use nemotron-3.5-lightning:30b-a3b-q4_K_M. E get 25 GB, e carry the full 1M maximum context, and na the build wey the latest, 30b and 30b-a3b pointers return when dem query am on October 25, 2026, even though the library move those pointers go separate digest for mid-October. Name am explicitly instead of pulling latest, so if dem republish that pointer later, e no go change your agent behaviour without you noticing. The mlx tags na Apple silicon builds, and 30b-a3b-nvfp4 sef na Apple silicon build, wey share the 30b-a3b-mlx digest. Dem no go help you for Linux.
How much RAM does Nemotron 3.5 Lightning need?
NVIDIA no publish minimum memory figure for the Ollama builds, so measure am instead of estimating. Pull the tag, run the model once, and read ollama ps while e dey loaded: e go print the size wey e actually occupy and whether e land for the GPU or CPU. The download size, 25 GB for the default tag, na just the floor, because KV cache dey add on top and e dey grow with the context window wey you set. If the plan too small, Ollama go refuse with model requires more system memory and name both numbers.
Can I run Nemotron 3.5 Lightning on a VPS with no GPU?
Yes, if the plan get enough RAM to hold the weights. The MoE design help because na about 3 out of the 30 billion parameters dem dey compute for each token. Speed na the main problem. Without GPU, memory bandwidth dey limit the model, so adding vCPUs barely change the result. Run ollama run --verbose with fixed prompt, read the eval rate line, and compare that number with your agent deadline. For batch job wey go run overnight, e often dey okay. For anything wey person dey wait for, e usually no dey okay.
Why does Ollama not give me the full 1M context window?
1M na the model maximum, no be Ollama default. Ollama dey apply much smaller window and discard the oldest tokens once conversation pass am. E no print any error, so e fit look like say the agent don forget its own instructions. Set OLLAMA_CONTEXT_LENGTH for the systemd service, or pass num_ctx for each request. Increase am small small and check ollama ps again every time, because KV cache memory dey scale with the window and fit push model layers comot from the GPU.
Is Nemotron 3.5 Lightning free to use commercially?
NVIDIA model card put the model under OpenMDW-1.1 license and mark am ready for commercial use, as checked on October 25, 2026. Ollama library page label the license shipped with its download as NVIDIA Open Model License Agreement, so read both before you depend on either one. This cover the weights wey you download and run by yourself. E no talk about the other software for your stack, so check the licences for the agent harness and any tools wey you connect to am separately. Also read the current model card before you depend on this for anything contractual.