Run the Flint coding model on a CPU VPS

Can you run the Flint coding model on a CPU VPS?

You can try to run the Flint coding model on a CPU VPS today, but it is new and the vendors do not promise that it works there yet. Flint is a mixture-of-experts (MoE) model from Recursive with 39.8B total parameters and about 2.5B active per token. A 4-bit build needs about 26 GB of RAM for the weights. It also needs a key-value (KV) cache for the context, so a 32 GB plan is the smallest the arithmetic allows.

As of 22 October 2026, here is where support stands. Ollama added Flint in v0.40.8, and those release notes say: "Flint is supported on MacOS. Other GPU support will be enabled in a subsequent release." They say nothing about Linux or CPU. llama.cpp merged Flint support on 21 October, in build b11803. The pull request author reports that the architecture test passes on CPU. The full-model runs listed there were on GPUs and an M4 Pro. We could not find a published full-model run on a CPU.

So this guide gives you two paths and the checks that tell you which one works on your server. It does not give a tokens-per-second figure, because we have not measured one on a CPU VPS. It gives you the procedure to measure your own.

What Flint is, from the model card

Recursive publishes Flint on its Hugging Face model card. It is a decoder-only MoE Transformer. The card gives the size as "39.8 billion total parameters and approximately 2.5 billion active per token". It adds that the active body alone is about 2.2B. The 2.5B figure includes the output head.

The model's shape, from the card and its config.json:

• 49 layers: 2 dense layers, then 47 MoE layers.
• Each MoE layer has 256 routed experts. The router picks 6 of them per token. One shared expert is always active as well.
• Attention uses 32 query heads, 4 KV heads and a head dimension of 128.
• 35 layers use sliding-window attention over 513 tokens. 14 layers use full attention.
• The context window is 262,144 tokens, and the prompt and the output share it.

The license is Apache 2.0. The card notes that Recursive's hosted API has its own terms, separate from the weight license. The full checkpoint is BF16 safetensors, about 80 GB, and the card sizes serving at "one B300 or H200 GPU per replica with the SGLang kit". SGLang is the serving path the card documents. The card also shows vLLM commands and says "Transformers modeling code is not included yet". It links community quantizations for llama.cpp, Ollama and LM Studio.

Recursive reports these scores for Flint: 54.9% on Terminal-Bench 2.1 (with the Terminus 2 harness), 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified (both with mini-swe-agent). The agentic scores are the average pass@1 over at least five attempts. These are Recursive's own claims. We did not reproduce them, and Ollama's model page calls them publisher-reported. A 4-bit quantization on a VPS processor is a different setup from the one that produced them.

Why 40B total and 2.5B active is a different CPU problem

A dense 40B model uses every weight for every token. Flint does not. For each token, each MoE layer runs 6 of its 256 experts plus the shared expert, so the model computes with about 2.5B parameters per token instead of 39.8B.

Two numbers matter here, and each one controls something different:

• Total parameters set the RAM. The router can pick any expert for the next token, so all 39.8B parameters must stay loaded. A plan with less RAM than the weights cannot hold the model.
• Active parameters set the speed. To generate a token on a CPU, the runtime reads the active weights from RAM. Flint reads a small part of its weights for each token, so it should generate much faster than a dense model with the same file size.

That is the whole case for Flint on a CPU server. You pay for RAM like a 40B model. If the runtime handles the expert routing well, you get speed closer to a small model. The "if" is real: the llama.cpp pull request itself notes a 10 to 30% slowdown from adaptive expert-activation scaling, to be fixed in later pull requests. For the general method of matching a model to a plan, see how big an LLM fits in your RAM. For another model built on the same small-active design, see running LFM2.5 24B-A2B on a VPS.

How much RAM does Flint need with the KV cache?

The weights are only one part. The KV cache holds the attention keys and values for every token in the context. It grows with the context length you allow. You can compute its size from config.json instead of guessing.

Only the 14 full-attention layers need a cache that grows with the context. For each token, those layers store a key and a value for each of the 4 KV heads. Each head has a dimension of 128, and each number takes 2 bytes in the default f16 format:

14 layers x 2 (K and V) x 4 KV heads x 128 x 2 bytes = 28,672 bytes per token

The 35 sliding-window layers keep only the last 513 tokens. If the runtime caps their cache at the window, they add about 35 MiB in total, whatever the context length. Recursive's llama.cpp card says to leave --swa-full unset to keep memory use down, and that keeps the cap in place.

The chart adds the cache to the Q4_1 weights. The 26.0 GB file is 24.2 GiB. These are estimates from the config. They leave out compute buffers, the operating system and your coding agent, so read them as a minimum.

[
  {
    "label": "32K context (32,768)",
    "kv_cache_gib": 0.875,
    "weights_plus_kv_gib": 25.1
  },
  {
    "label": "64K context (65,536)",
    "kv_cache_gib": 1.75,
    "weights_plus_kv_gib": 26.0
  },
  {
    "label": "128K context (131,072)",
    "kv_cache_gib": 3.5,
    "weights_plus_kv_gib": 27.7
  },
  {
    "label": "256K context (262,144)",
    "kv_cache_gib": 7.0,
    "weights_plus_kv_gib": 31.2
  }
]

At 64K the cache is 1.75 GiB, which puts weights plus cache at 26.0 GiB. On a 32 GB plan that leaves a few GiB for everything else. At the full window the cache alone is 7.0 GiB and the total reaches 31.2 GiB. That does not fit a 32 GB plan once the system and the agent need memory too, so 256K is not a VPS setting. Pick a context, set it yourself, and then check the real figure with free -h and ollama ps.

If you want to run an agent, aim for 64K. Ollama's opencode page says "OpenCode requires a context length of 64k or higher". Flint's card asks for an output budget of at least 32,768 tokens on hard problems, reasoning included. At 64K, half the window can go to the answer.

Which Flint tag to pull on a CPU VPS

On 22 October 2026, Ollama's tags page for flint lists seven tags:

• flint:latest, flint:40b and flint:40b-a2b: one build, digest 905c74dfd56b, 23GB.
• flint:40b-a2b-gguf-nvfp4: digest 39fc172e198d, 23GB, marked MLX.
• flint:40b-a2b-qat-int4: digest a503b9032955, 23GB.
• flint:40b-a2b-q4_1: digest f7d42a0a1137, 26GB.
• flint:40b-a2b-mlx-bf16: digest 105775bdd1a7, 80GB, marked MLX.

The default tag is not the obvious choice for a CPU. Its registry manifest shows that its model layer was built from a file named flint-nvfp4.gguf. NVFP4 is NVIDIA's 4-bit floating-point format, and we have not tested how Ollama's CPU backend handles it. The MLX tags are for MLX, Apple's machine learning framework. The q4_1 tag uses Q4_1, a long-standing GGUF quantization type. It also matches the format of the GGUF that Recursive publishes for llama.cpp. On a CPU VPS, start with flint:40b-a2b-q4_1. To see what Q4 gives up compared with Q8 and FP16, read Ollama quantization levels compared.

Path A: install a pinned Ollama and try Flint

Pin the Ollama release. v0.40.8 is the first release whose notes announce Flint. The install script reads the OLLAMA_VERSION variable:

curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.40.8 sh
ollama --version
systemctl status ollama --no-pager

ollama --version should print 0.40.8, and the service should show active (running). A later release may add Linux support for Flint. If it does, its release notes will say so, and that is the version to pin next.

Pull the Q4_1 tag and record exactly what you got:

ollama pull flint:40b-a2b-q4_1
ollama list

Copy the ID that ollama list prints for the tag into your notes. On 22 October 2026 the tags page showed f7d42a0a1137 for this tag. If your ID is different, the tag was updated after that date, so your notes should say which build you tested. To learn where the 26GB goes on disk, see ollama pull vs run, and where models are stored.

Cap the context with a Modelfile

Do not rely on a default context. Ollama's context-length page sets the default by VRAM, and it does not say what applies on a server with no GPU. The Modelfile page lists a different default of 2048. Set the value yourself, in the model:

mkdir -p ~/flint && cd ~/flint
printf 'FROM flint:40b-a2b-q4_1\nPARAMETER num_ctx 65536\n' > Modelfile
ollama create flint-64k -f Modelfile

Put the context in the model, not in the client. Ollama's OpenAI compatibility page says: "The OpenAI API does not have a way of setting the context size for a model." A coding agent that talks to /v1 cannot fix a small context for you. The other option is OLLAMA_CONTEXT_LENGTH=65536 in a systemd override, which applies to the whole server. Create the override with sudo systemctl edit ollama.service.

Load it and read the result

ollama run flint-64k "Write a bash one-liner that counts lines in all .py files under the current directory."
ollama ps
free -h
journalctl -e -u ollama

A working load prints an answer. ollama ps should then list flint-64k with a context of 65536, running on the CPU. free -h shows how much memory the load really used. Compare it with the chart.

If the run fails, read the end of journalctl -e -u ollama. The log line there gives the real reason, so read it before you guess. The v0.40.8 notes name macOS as the supported platform, so a failure on Linux is a likely outcome until a later release says otherwise. If that happens, use Path B.

Measure tokens per second yourself

The /api/generate response includes timing fields. eval_count is the number of output tokens, and eval_duration is the time spent generating them, in nanoseconds. The prompt has matching prompt_eval_count and prompt_eval_duration fields.

sudo apt install -y jq
curl -s http://127.0.0.1:11434/api/generate \
  -d '{"model":"flint-64k","prompt":"Explain what a Linux inode is in three sentences.","stream":false}' |
  jq '{prompt_tokens: .prompt_eval_count, prompt_tok_per_s: (.prompt_eval_count / .prompt_eval_duration * 1e9), gen_tokens: .eval_count, gen_tok_per_s: (.eval_count / .eval_duration * 1e9), load_s: (.load_duration / 1e9)}'

Run it three times. The first run includes the model load, which load_s shows. Report the later runs. Report prompt speed and generation speed separately, because an agent sends long prompts and both numbers decide how long you wait. The method and its traps are covered in measuring tokens per second on a local LLM.

Path B: run Flint with llama.cpp on the CPU

llama.cpp lists Flint support from build b11803. Pin that build as the minimum. A newer build may include the speed fixes the pull request promised, so write down the exact build you test.

sudo apt update && sudo apt install -y build-essential cmake git libssl-dev
git clone --depth 1 --branch b11803 https://github.com/ggml-org/llama.cpp ~/llama.cpp
cd ~/llama.cpp
cmake -B build
cmake --build build --config Release -j "$(nproc)"
./build/bin/llama-server --version

Download Recursive's Q4_1 GGUF with the Hugging Face CLI. If the shell cannot find hf after the installer finishes, open a new shell.

curl -LsSf https://hf.co/cli/install.sh | bash
hf download RecursiveSI/flint-int4-gguf flint-int4-q4_1.gguf --local-dir ~/flint-int4-gguf

Know what this card targets. It says the file is "primarily intended for Apple Silicon using llama.cpp with Metal". It requires a Mac with at least 48 GB of unified memory, and it gives no CPU guidance. The command below is the card's server command with the Apple-specific parts removed. It has no -ngl all, because there is no GPU to offload layers to. It drops the Metal batch tuning, and it uses a 64K context instead of the full window.

~/llama.cpp/build/bin/llama-server -m ~/flint-int4-gguf/flint-int4-q4_1.gguf \
  --alias flint -c 65536 -np 1 --cache-ram 0 \
  -t "$(nproc)" -ctk f16 -ctv f16 --jinja \
  --temp 1 --top-p 0.95 --top-k 20 --min-p 0 \
  --host 127.0.0.1 --port 8080

The sampling flags match the card: temperature 1.0, top-p 0.95 and top-k 20. --jinja makes the server use the chat template stored in the GGUF, which the card asks for. In a second shell, curl -s http://127.0.0.1:8080/v1/models should list flint. If the server exits during the load instead, the last lines it printed give the reason. To decide which runtime suits you, see Ollama vs llama.cpp.

Keep the model API on localhost

Ollama binds 127.0.0.1:11434 by default. The llama-server command above binds 127.0.0.1:8080. Keep both on that address. Anyone who can reach the port can run the model and use your CPU. Check that neither one listens on a public address:

ss -ltnp | grep -E ':(11434|8080)'

Both lines should show 127.0.0.1. Do not set OLLAMA_HOST=0.0.0.0 to reach the model from a laptop. Use an SSH tunnel instead, which keeps the port closed: ssh -L 11434:127.0.0.1:11434 you@your-vps.

Wire Flint to a coding agent, and test tool calling first

Flint's card says "Flint supports OpenAI-compatible tool calls" and multi-turn tool use. Ollama tags the model with tools. Neither one proves that a tool call survives the Ollama /v1 path on your build. Test it with one request before you start an agent:

curl -s http://127.0.0.1:11434/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "flint-64k",
  "messages": [{"role": "user", "content": "List the files in /etc/nginx."}],
  "tools": [{"type": "function", "function": {"name": "list_files", "description": "List files in a directory", "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}}]
}' | jq '.choices[0].message | {tool_calls, content}'

A pass shows a tool_calls array with "name": "list_files" and arguments that contain /etc/nginx. A failure shows tool_calls: null, with the call written out as text in content. That means the agent gets text where it expects an action. For Path B, send the same request to port 8080 with "model": "flint".

Next, install opencode on the same server. The full setup is in running opencode on a VPS.

curl -fsSL https://opencode.ai/install | bash

opencode connects to a local model through a custom provider built on @ai-sdk/openai-compatible. Put this in ~/.config/opencode/opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": { "baseURL": "http://127.0.0.1:11434/v1" },
      "models": {
        "flint-64k": { "name": "Flint 64K" }
      }
    }
  }
}

For llama-server, set baseURL to http://127.0.0.1:8080/v1 and name the model flint. Then add "limit": { "context": 65536, "output": 32768 } to that model. Recursive's own opencode.json assumes the full 262,144-token context. Its card says to change the client limit to match when you change -c. A client that believes it has 256K will send prompts that the server cannot hold.

The two projects give different advice on context. opencode's provider docs say: "If tool calls aren't working, try increasing num_ctx in Ollama. Start around 16k - 32k." Ollama's opencode page asks for 64K. Use 64K. An agent sends its system prompt, its tool definitions and file contents on every turn, so a small window fills before the work starts. Other agent clients are covered in using Ollama with your coding agent.

Two more points from the card matter for an agent. First, it says to "preserve returned reasoning_content in conversation history". We have not checked whether opencode sends that field back through this provider. If multi-step tasks go wrong after the first tool call, check this first. Second, the card says that "structured tool-call grammar does not prevent the model from choosing an unsafe or incorrect action." Run the agent as an ordinary user, inside a project directory, with your own limits on what it may run.

What Flint on a CPU VPS does not replace

• The setup behind the benchmark scores. Recursive measured its own configuration. A 4-bit quantization on a shared VPS processor is a different system, so test it on your own tasks before you trust it.
• A GPU server for long agent runs. The card sizes Flint for one H200 or B300 per replica. On a CPU, you only learn how fast the model reads a long agent prompt by measuring it.
• A permission system. The card says plainly that the grammar does not make actions safe. Your sandbox and your own review still do that job.
• A stable platform, for now. On 22 October 2026, Ollama names macOS as its supported platform for Flint, and llama.cpp support is one day old. Pin versions, record digests, and test again when either project ships a release that mentions Flint.

You may want a local model for routine turns and a hosted one for hard ones. Routing between models in a coding agent covers that split.

FAQ

Does Ollama run Flint on Linux without a GPU?

Ollama has not said that it does. The v0.40.8 release notes, which added Flint, say: "Flint is supported on MacOS. Other GPU support will be enabled in a subsequent release." They do not mention Linux or CPU. Try it with ollama run, then check ollama ps and journalctl -e -u ollama. If the load fails, switch to llama.cpp build b11803 or later.

How much RAM does the Flint coding model need on a VPS?

All 39.8B parameters must stay in memory, because the router can pick any expert for the next token. The Q4_1 file is 26.0 GB, about 24.2 GiB. At a 64K context, the full-attention KV cache adds about 1.75 GiB, computed from config.json. That makes a 32 GB plan the smallest that fits, before you add the operating system and an agent. Check the real figure with free -h after the model loads.

Why should Flint be faster than a dense 40B model on a CPU?

For each token, Flint runs only 6 of its 256 routed experts in each MoE layer, plus one shared expert. That is about 2.5B parameters instead of 39.8B, so it reads far fewer weights from RAM for each token. No published CPU speed figure exists for Flint. Measure your own with the eval_count and eval_duration fields from Ollama's /api/generate response.

Are Flint's SWE-bench and Terminal-Bench scores reliable?

They are Recursive's own claims: 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified, each the average pass@1 over at least five attempts. Ollama's page calls them publisher-reported. They were not measured on a 4-bit build on a CPU VPS. Test the model on your own repository before you rely on them.

Why do tool calls from opencode fail with Flint?

Check the context first. The OpenAI-compatible endpoint cannot set it, so it must come from num_ctx in a Modelfile or from OLLAMA_CONTEXT_LENGTH, and Ollama asks for 64K for opencode. Then send one /v1/chat/completions request with a tools array. If the reply has no tool_calls and the call appears as text in content, tool calling does not survive that path on your build.