Run LFM2.5-24B-A2B on a CPU-Only VPS

What LFM2.5-24B-A2B needs from a CPU-only VPS

You can run LFM2.5-24B-A2B on a CPU-only VPS with llama.cpp's llama-server, as long as the box has enough RAM for the whole model. The Q4_K_M file is 14.5 GB, and for usable speed all of it should stay resident in memory. llama.cpp memory-maps the file, so it can start with less RAM, but any part that is not in RAM has to be read back from disk when it is needed. Once the model fits, memory bandwidth sets the speed. The number of cores you rent matters much less. This guide builds a pinned llama.cpp from source, downloads the model, applies Liquid's recommended sampling settings, and keeps the API on localhost.

LFM2.5-24B-A2B is a model from Liquid AI. The model card describes it as "a 24B-parameter reasoning model with only 2B active parameters, built for fast, private agents on-device". It is a MoE (mixture of experts) model. The card's specification list gives 2.45B active parameters and 40 layers, of which 30 are short convolution blocks and 10 are GQA (grouped-query attention) layers. Each MoE block has 64 experts, and top-4 routing picks four of them for each token. The context length is 131,072 tokens.

Why a mixture-of-experts model behaves differently on a CPU

A dense model reads every weight for every token it generates. A MoE model does not. For each token, a router in each MoE layer picks 4 of the 64 experts, and only those experts do any work. That is why the card can say 24B total but about 2.45B active.

This has two effects on a CPU server, and they pull in opposite directions.

First, all 24B parameters must stay reachable, and for good speed they belong in RAM. The router picks different experts for different tokens, so you cannot know ahead of time which experts the next token needs. llama.cpp memory-maps the model by default, and its documentation says this lets the system load "only the necessary parts of the model as needed". So an expert that is not in RAM does not stop the model. It is read from disk when the router picks it. A disk read in the middle of a token is far slower than a RAM read, though, so give the box enough RAM to keep the whole file resident.

Second, only the active part is read for each token. Generating one token means streaming the active weights from RAM into the CPU. On a CPU, that memory traffic is the slow part, not the arithmetic. Our guide to how big an LLM fits in your RAM puts it in one line: "the ceiling on tokens per second is your memory bandwidth divided by the active weight bytes per token". The active share here is about a tenth of the model (2.45B of 24B). So a CPU box generates much faster with this model than with a dense 24B model of the same file size.

The practical result: buy RAM capacity to fit the model, and judge speed by memory bandwidth. Extra cores help only until they saturate the memory bus. After that point they wait for data like the rest.

How much RAM does LFM2.5-24B-A2B need?

Liquid publishes these GGUF (the file format llama.cpp loads) sizes on the model card. The sizes are decimal gigabytes, as the card lists them.

[
  {
    "label": "QAD-Q4_0",
    "size_gb": 13.6
  },
  {
    "label": "Q4_K_M",
    "size_gb": 14.5
  },
  {
    "label": "Q5_K_M",
    "size_gb": 17
  },
  {
    "label": "Q6_K",
    "size_gb": 19.7
  },
  {
    "label": "Q8_0",
    "size_gb": 25.5
  },
  {
    "label": "BF16",
    "size_gb": 48
  },
  {
    "label": "F16",
    "size_gb": 48
  }
]

The file size is the floor, not the total. Three more things need memory on top of it:

• The KV cache (the stored attention keys and values for every token in the context). It grows with the context size you set.
• The operating system and any other services on the box.
• Headroom, so the kernel never has to evict model pages and read them back from disk.

Only 10 of the 40 layers are attention layers, so the KV cache is smaller than a pure-attention model of the same depth would need. It is still not zero. Use the KV formula in the RAM guide with the model's own config values if you plan to run long contexts.

A simple sizing rule follows from the table. The Q4_K_M file is 14.5 GB, so a 16 GB VPS leaves almost nothing for the KV cache and the OS. Plan on 24 GB or more for Q4_K_M with a moderate context. The Q8_0 file is 25.5 GB, which is about 23.7 GiB. A VPS with 32 GiB of RAM therefore leaves roughly 8 GiB for the KV cache, the OS and headroom. That can be enough for a moderate context, but it is tight, so pick a larger plan if you want long contexts at Q8_0. The card also lists a QAD-Q4_0 file at 13.6 GB. Liquid trained it to recover the accuracy that plain Q4_0 loses. If you want to understand the trade between these levels, our comparison of Q4 versus Q8 versus FP16 quantization explains what each step costs in quality and memory.

Check what your box actually has before you download anything:

free -g
df -h ~
nproc

free -g shows total and available RAM. df -h ~ shows free disk in your home directory. You need room for the model file plus the llama.cpp build.

Pin a llama.cpp release that knows this architecture

The model's config.json names the architecture Lfm2MoeForCausalLM. llama.cpp supports this family under the architecture name lfm2moe. Support first landed in pull request #16464 ("llama : support LiquidAI LFM2-MoE hybrid model"), merged on 2025-10-07. A fix for missing tensors followed in pull request #18132, merged on 2025-12-17.

This guide pins llama.cpp to tag v0.7.0, published on 2026-10-15 at commit 620d2225cc436ab0b001e3200e590e3b286ef9ab. That tag contains both of those commits. The v0.7.0 release ships no prebuilt server binary, so you build it from source. Pinning to a tag means a later upstream change cannot silently alter the server under you. The reasons, and how to move to a newer tag safely, are in our guide to pinning llama.cpp releases on a server.

Install the build tools, then clone exactly that tag:

sudo apt update
sudo apt install -y build-essential cmake git curl jq libssl-dev
git clone --branch v0.7.0 --depth 1 https://github.com/ggml-org/llama.cpp.git ~/llama.cpp
git -C ~/llama.cpp log -1 --format=%H

The last command should print 620d2225cc436ab0b001e3200e590e3b286ef9ab. Any other hash means you are not on v0.7.0.

Now build only the server:

cd ~/llama.cpp
cmake -B build
cmake --build build --config Release -j "$(nproc)" --target llama-server
ls -l build/bin/llama-server

The build takes several minutes on a small VPS. The last command should list the binary. If cmake -B build stops with an error, read the first error line. A missing compiler or a missing cmake means the apt install step did not complete.

Download the GGUF file

Download the Q4_K_M file directly from Liquid's GGUF repository. A plain curl download works on any build and leaves you with a normal file you can back up or move.

mkdir -p ~/models
curl -L --fail -o ~/models/LFM2.5-24B-A2B-Q4_K_M.gguf https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF/resolve/main/LFM2.5-24B-A2B-Q4_K_M.gguf
stat -c %s ~/models/LFM2.5-24B-A2B-Q4_K_M.gguf
sha256sum ~/models/LFM2.5-24B-A2B-Q4_K_M.gguf

stat should print 14526258464, the size in bytes. That is about 13.5 GiB, which is the same file as the card's 14.5 GB in decimal units. Compare the sha256sum output with the SHA-256 shown on the file's page on Hugging Face. A smaller size or a different hash means the download was cut short, and llama-server will fail to load the file.

To use the QAD-Q4_0 file instead, change both file names to LFM2.5-24B-A2B-QAD-Q4_0.gguf.

Start llama-server on localhost

Start the server in the foreground first, so you can read the load log:

~/llama.cpp/build/bin/llama-server -m ~/models/LFM2.5-24B-A2B-Q4_K_M.gguf --host 127.0.0.1 --port 9931 -c 16384 --temp 0.1 --top-k 50 --repeat-penalty 1.05

Each flag has a reason.

• --host 127.0.0.1 keeps the API on the loopback interface. This is already the default in v0.7.0, but writing it down protects you if a default changes.
• --port 9931 is the v0.7.0 default. Older guides use 8080. Pass the port explicitly so your scripts do not depend on the version.
• -c 16384 sets the context size. The default of 0 means the server takes the context length from the model, which is 131,072 tokens here. A larger context means a larger KV cache, so set only what you need.
• --temp 0.1 --top-k 50 --repeat-penalty 1.05 are the sampling values the card recommends. The server's generic defaults are a temperature of 0.80, top-k of 40 and no repeat penalty, so pass these three flags yourself.

While the model loads, the log prints the model's metadata, including its architecture name. You should see the lfm2moe family there. An error about an unknown model architecture means the binary you are running is not the one built from the pinned tag.

The server has no API key by default. On loopback that is fine, because only processes on the box can connect. To use it from your laptop, open an SSH tunnel instead of opening a port:

ssh -L 9931:127.0.0.1:9931 user@your-vps-ip

Your laptop can then reach the API at http://127.0.0.1:9931. If you must put the server behind a reverse proxy, add --api-key with a long random value. Note that /health is public by design and does not check the key. Our guide to running llama.cpp's server on a VPS covers the wider setup around the server process.

Check that the model loaded and answers

In a second SSH session, ask the health endpoint for its status code:

curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:9931/health

503 means the model is still loading. 200 means the server is ready. Loading 14.5 GB from disk takes a while on network storage, so wait and repeat.

Then send a real request to the OpenAI-compatible endpoint:

curl -s http://127.0.0.1:9931/v1/chat/completions -H "Content-Type: application/json" -d '{"messages":[{"role":"user","content":"Reply with the single word: ready"}],"max_tokens":1024}' | jq '.choices[0].message, .timings.predicted_per_second'

The card says the model always reasons before it answers. Expect reasoning text before the final word, and leave room for it in max_tokens. A low limit cuts the reply off inside the reasoning, so you never see the answer.

Measure your own tokens per second

This guide prints no tokens-per-second figure for a VPS. Speed depends on your plan's memory bandwidth, so a number from another box tells you little about yours. Liquid's own numbers are from desktop-class hardware, not from any VPS:

[
  {
    "label": "AMD Ryzen AI Max+ 395",
    "decode_tok_s": 114
  },
  {
    "label": "Apple M5 Max",
    "decode_tok_s": 184
  }
]

Liquid reports 114 tokens per second on an AMD Ryzen AI Max+ 395 and 184 on an Apple M5 Max. Both machines have far more memory bandwidth than a typical VPS. Treat them as an upper reference, not as a forecast.

Get your own number from the server itself. Every /v1/chat/completions response carries a timings object, and its predicted_per_second field is the generation speed for that request. The jq command above already prints it. Run the same prompt five times and look at the spread. For a broader method, including when a rented GPU starts to beat per-token API billing, read our guide to measuring tokens per second on a local LLM.

What LFM2.5-24B-A2B is good at, and tool calling

The card names agentic work and tool use as the model's strengths. It also says the model is "not recommended for knowledge-heavy tasks on its own", and suggests pairing it with retrieval or web search. In practice: give it tools and documents, and do not ask it to recall facts from memory. A search backend such as a SearXNG search skill for agents fits that pattern.

By default the model writes tool calls as Pythonic function calls wrapped in special tokens, such as <|tool_call_start|>[get_candidate_status(candidate_id="12345")]<|tool_call_end|>. You can ask for JSON calls instead in the system prompt. The v0.7.0 server supports OpenAI-style function calling through its Jinja chat template engine, which is on by default. Whether the server turns this model's format into a structured tool call depends on its parser for this template. Test it before you build on it: send a request with a tools array, then look at the response. A structured tool call in the message means parsing works. The raw <|tool_call_start|> text inside content means your agent code must parse the call itself.

Does the Ollama command on the model card work?

The GGUF card gives this command for Ollama:

ollama run hf.co/LiquidAI/LFM2.5-24B-A2B-GGUF:Q4_K_M

We checked the download step, not the full load. Hugging Face's Ollama-style manifest for the :Q4_K_M tag resolves without error. It lists one model layer of 14,526,258,464 bytes, and its hash matches the Q4_K_M file. So the pull should fetch the right file. We did not run the model in Ollama, so we cannot tell you that it loads and generates correctly. Ollama's latest release notes at the time of writing do not mention LFM2.5, and Ollama has open pull requests about parsing LFM2 tool calls. If you rely on tool calling, test it in Ollama before you depend on it.

Do not confuse this with lfm2:24b in Ollama's own library. That entry is the older LFM2-24B-A2B, listed with a 32K context window. It is a different model. For the general trade between the two runtimes, see Ollama versus llama.cpp.

The license: check it before production use

The model ships under the LFM Open License v1.0, not under Apache or MIT. The license defines a revenue "Threshold" of about 10 million US dollars per year. Commercial use by a legal entity above that threshold is not licensed. A qualified non-profit is exempt from the threshold, but only for non-commercial or research purposes. The wording at exactly 10 million dollars is not fully consistent, so read the LICENSE file in the model repository yourself before you put this model into a commercial product. This section is a summary, not legal advice.

Run llama-server as a systemd service

A server started in an SSH session stops when the session ends. Run it under systemd so it starts at boot and restarts after a crash. Create /etc/systemd/system/llama-lfm.service, replacing youruser with your login name:

[Unit]
Description=llama-server with LFM2.5-24B-A2B
After=network.target

[Service]
User=youruser
ExecStart=/home/youruser/llama.cpp/build/bin/llama-server -m /home/youruser/models/LFM2.5-24B-A2B-Q4_K_M.gguf --host 127.0.0.1 --port 9931 -c 16384 --temp 0.1 --top-k 50 --repeat-penalty 1.05
Restart=on-failure

[Install]
WantedBy=multi-user.target

Then load and start it:

sudo systemctl daemon-reload
sudo systemctl enable --now llama-lfm
systemctl status llama-lfm
journalctl -u llama-lfm -n 50

systemctl status should show active (running). If it shows a failed state, journalctl prints the load log, and the first error line names the cause. A wrong path in ExecStart is the most common one. Then repeat the health check until it returns 200.

FAQ

How much RAM do I need to run LFM2.5-24B-A2B on a VPS?

The Q4_K_M file is 14.5 GB on the model card, and the whole file should fit in RAM because any of the 64 experts can be picked for the next token. llama.cpp memory-maps the model, so it can still run with less, but any expert that is not in RAM is read from disk when it is picked, and that is far slower. The KV cache, the operating system and some headroom come on top. A 16 GB VPS is too tight for Q4_K_M. Plan on 24 GB or more, and set a modest context with -c so the KV cache stays small.

Why is a 24B MoE model usable on a CPU at all?

Only about 2.45B of the 24B parameters are active for each token, so the CPU reads roughly a tenth of the weights per generated token. On a CPU, reading weights from RAM is the slow step, so speed follows memory bandwidth divided by the active bytes. A dense 24B model reads all of its weights for every token and is much slower on the same box.

Which llama.cpp version supports LFM2.5-24B-A2B?

llama.cpp supports this family under the lfm2moe architecture, first added in pull request #16464 in October 2025, with a tensor fix in pull request #18132 in December 2025. This guide pins tag v0.7.0, which contains both. The v0.7.0 release has no prebuilt binary, so build llama-server from the tag, then confirm the architecture line in the load log.

Why does my curl request to port 8080 fail?

In llama.cpp v0.7.0 the server's default port is 9931, not the 8080 that older guides use. Nothing is listening on 8080, so the connection is refused. Pass --port explicitly when you start the server, and use the same port in every client.

Can I use LFM2.5-24B-A2B commercially?

The LFM Open License v1.0 allows commercial use below a revenue threshold of about 10 million US dollars per year. Commercial use by a company above that threshold is not licensed. Non-profits are exempt from the threshold only for non-commercial or research use. Read the LICENSE file in the model repository before production use, and ask a lawyer if your case is close to the line.