By Run LFM2.5-230M locally on a CPU or small VPS
Serve Liquid AI's 230M model with a pinned llama-server build on a 1 or 2 vCPU box, then test where it extracts clean JSON and where its reasoning fails.
What you get when you run LFM2.5-230M locally
To run LFM2.5-230M locally, download one GGUF file of about 150 MB and serve it with llama-server from a pinned llama.cpp release. You do not need a GPU, and one or two vCPUs are enough to try it. The model card aims it at narrow jobs such as pulling fields out of text. It warns against tasks that need several steps of reasoning.
LFM2.5-230M is a text-only model from Liquid AI. It has 230 million parameters and a context of 32,768 tokens, according to its model card. The card says it was distilled from LFM2.5-350M and then refined with multi-stage reinforcement learning. Distillation means a smaller model is trained to copy the outputs of a larger one.
The card states the intended use plainly: "We recommend using it for data extraction and lightweight on-device agentic pipelines." It states the limit just as plainly: "It is not recommended for reasoning-heavy workloads such as advanced math, code generation, or creative writing." The card also documents function calling. By default the model writes Python-style calls between <|tool_call_start|> and <|tool_call_end|> tokens, and a system prompt can ask for JSON calls instead.
This guide installs a pinned llama.cpp build on Ubuntu 24.04, downloads the model, measures speed and memory on your own box, and then runs two tests. The first is a JSON extraction job that matches the card's intended use. The second is a short reasoning task that falls inside the card's warning.
Do not confuse it with LFM2.5-Encoder-230M
Liquid AI publishes a second model with almost the same name. LFM2.5-Encoder-230M is a different kind of model. Its card describes the family as "multilingual bidirectional encoders built on the LFM2 architecture". It is a masked language model, which means it predicts hidden words inside a sentence instead of writing text left to right. The card offers it as a base to fine-tune into "a backbone for dense embedding or late-interaction (ColBERT-style) retrievers". It is not a ready-made embedding model, and it is not a chat model. Its context is 8,192 tokens.
For chat or extraction, the repository you want is LiquidAI/LFM2.5-230M-GGUF. If you want embeddings for search today, start with a model trained for that job, for example by running EmbeddingGemma 2 as an embedding server on a VPS.
Which GGUF file should you download?
GGUF is the single-file model format that llama.cpp reads. Liquid AI and unsloth both publish LFM2.5-230M in this format, and Hugging Face lists many more community GGUF conversions of the same model. Liquid AI's own repository has Q4_0, Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16 and BF16 files, plus LFM2.5-230M-QAD-Q4_0.gguf. The unsloth repository has more than 20 GGUF files, from UD-IQ2_M up to BF16. The unsloth card's example commands use UD-Q4_K_XL. The card does not call that file the recommended one, so treat it as one more option.
The data behind this chart
[
{
"label": "LiquidAI Q4_0",
"download_mb": 149
},
{
"label": "LiquidAI Q4_K_M",
"download_mb": 153
},
{
"label": "unsloth UD-Q4_K_XL",
"download_mb": 160
},
{
"label": "LiquidAI Q5_K_M",
"download_mb": 172
},
{
"label": "LiquidAI Q6_K",
"download_mb": 191
},
{
"label": "LiquidAI Q8_0",
"download_mb": 247
},
{
"label": "LiquidAI BF16",
"download_mb": 462
}
]Every file in the range fits easily in the RAM of a 1 GB VPS. Even the BF16 file is only 462 MB, so you are choosing on speed and quality, not on memory. This guide uses Liquid AI's Q4_K_M at 153 MB. It is the vendor's own file, and it is the quant that llama.cpp's -hf flag and Hugging Face's Ollama route both pick by default. Q8_0, at 247 MB, is the file to try if the Q4_K_M output looks worse than you expect. For the general rule on how parameter count and quantization turn into memory, see how big an LLM fits in your RAM.
The QAD file needs one note. QAD (quantization-aware distillation) means the 4-bit version was trained to match the full model, instead of being rounded after training. The Liquid AI card says this file is distinct from the post-training-quantized LFM2.5-230M-Q4_0.gguf, although both use the GGUF Q4_0 format. It also says the published QAD results apply to that Q4_0 GGUF only. Test it against Q4_K_M on your own prompts.
Install a pinned llama.cpp release
llama.cpp publishes a new release tag many times a day. A server that pulls "latest" on every rebuild changes under you without warning, so pin one tag and upgrade on purpose. The reasons, and a clean upgrade routine, are in pinning llama.cpp releases on a server. This guide pins b12189, the newest tag on 7 November 2026. Replace it with the tag you choose, and use the same tag everywhere below.
sudo apt update && sudo apt install -y curl jq libgomp1
TAG=b12189
sudo mkdir -p /opt/llama /opt/models
cd /opt/llama
sudo curl -LO https://github.com/ggml-org/llama.cpp/releases/download/$TAG/llama-$TAG-bin-ubuntu-x64.tar.gz
sudo tar xzf llama-$TAG-bin-ubuntu-x64.tar.gz
ls llama-$TAG
./llama-$TAG/llama-server --versionThe release workflow packs the build's bin directory into a folder named after the tag, so the files land in /opt/llama/llama-b12189/. The listing should include llama-server and llama-bench, plus the shared libraries they load. The build looks for those libraries in its own folder, so keep the extracted folder together. Do not copy only the binary somewhere else. On an ARM VPS, download llama-$TAG-bin-ubuntu-arm64.tar.gz instead.
The upstream README now shows a llama serve command, and some model cards use it too. The server's own README still documents the llama-server binary, and that is the binary this guide calls.
Download the model and check the file
sudo curl -L -o /opt/models/LFM2.5-230M-Q4_K_M.gguf https://huggingface.co/LiquidAI/LFM2.5-230M-GGUF/resolve/main/LFM2.5-230M-Q4_K_M.gguf
ls -l /opt/models/LFM2.5-230M-Q4_K_M.gguf
sha256sum /opt/models/LFM2.5-230M-Q4_K_M.ggufOn 7 November 2026, Hugging Face listed this file at 153406304 bytes with SHA-256 7bbd90384d3deffe4c646ec9643b212802d32d4ce417c90a1ec9282100650062. If Liquid AI uploads a new version, both values change, so compare them with the file page and not only with this guide. A file of a few hundred bytes means you saved an error page or a Git LFS pointer instead of the model. Check that the URL contains /resolve/main/.
Measure tokens per second on your own box
The card gives two speed figures. Both come from Liquid AI's own devices, not from a VPS.
The data behind this chart
[
{
"device": "Raspberry Pi 5",
"vendor_tok_s": 42
},
{
"device": "Galaxy S25 Ultra (decode)",
"vendor_tok_s": 213
}
]The card reports 42 tokens per second on a Raspberry Pi 5 and 213 on a Galaxy S25 Ultra. It does not state the quantization or the thread count. A VPS vCPU is a share of a server core, and its speed depends on the CPU model and on other tenants of the same host. Measure your own box instead of copying these figures. llama-bench ships in the same tarball and tests several thread counts in one run.
nproc
lscpu | grep 'Model name'
/opt/llama/llama-b12189/llama-bench -m /opt/models/LFM2.5-230M-Q4_K_M.gguf -t 1,2 -p 512 -n 128nproc prints how many vCPUs you have. On a 1 vCPU box, use -t 1 only, because a second thread has no core of its own to run on. The output has one row per test and thread count. The pp512 rows measure prompt processing over a 512-token prompt. The tg128 rows measure text generation over 128 new tokens. The t/s column is the average tokens per second, followed by the standard deviation across five repetitions.
The tg128 figure is the one a user feels, because it is the speed at which the answer appears. The pp512 figure matters for extraction, because a long input document is mostly prompt. Write down the CPU model, the thread count, the quant file and the llama.cpp tag next to each result. A number without those four facts cannot be compared with anything. The method behind these measurements, and the mistakes that inflate them, are covered in how to measure tokens per second for a local LLM.
Start llama-server with the card's sampling settings
The card recommends temperature 0.1, top_k 50 and a repetition penalty of 1.05. The llama-server defaults are different: temperature 0.80, top-k 40, and a repeat penalty of 1.00, which means disabled. Pass the card's values as startup flags.
/opt/llama/llama-b12189/llama-server -m /opt/models/LFM2.5-230M-Q4_K_M.gguf -c 8192 -np 1 -t 2 --host 127.0.0.1 --port 9931 --temp 0.1 --top-k 50 --repeat-penalty 1.05Three flags need an explanation. -c sets the context size. Its default is 0, which means "use the value stored in the model file", and the GGUF metadata stores 128000, not the 32,768 the card states. Set -c yourself and keep it at 32768 or below, because the card does not promise a longer context. 8192 is enough for most extraction jobs and reserves less memory. -np 1 sets one server slot. The default is automatic, and setting 1 makes the slot count explicit on a box this small. The startup log shows the slot count it used. -t is the thread count that won in your benchmark.
The server listens on port 9931 by default, and --host 127.0.0.1 keeps it off the public internet. Older guides say port 8080. Check that the server is ready from a second SSH session:
curl -s http://127.0.0.1:9931/health
ps -C llama-server -o rss=,args=/health returns HTTP 503 while the model loads and {"status":"ok"} when it is ready. The first number from ps is the RSS (resident set size) of the server in KiB, which is the RAM it holds right now. Stop the server, start it again with -t 1, and run ps again to see whether memory changes with the thread count on your build. Record the RSS next to your speed results. For a full walk-through of the server, its API and its other flags, see running llama.cpp server on a VPS.
A JSON extraction task it should handle
This is the job the card recommends. The request sends a short invoice text and a JSON schema. llama-server turns the schema into a grammar, so at each step the sampler may only pick tokens that keep the output valid against the schema. The result is parseable JSON with the fields you asked for, unless the reply hits max_tokens first. The schema does not make the values correct, so you still have to check them.
cat > /tmp/extract.json <<'EOF'
{
"messages": [
{"role": "system", "content": "Extract the invoice fields from the user's text. Reply with JSON only."},
{"role": "user", "content": "Invoice INV-2291 from Northwind Supplies, dated 3 October 2026. Total due: 1,480.00 EUR. Payment within 30 days."}
],
"max_tokens": 200,
"response_format": {
"type": "json_schema",
"json_schema": {
"schema": {
"type": "object",
"properties": {
"invoice_number": {"type": "string"},
"vendor": {"type": "string"},
"date": {"type": "string"},
"total": {"type": "number"},
"currency": {"type": "string"},
"payment_days": {"type": "integer"}
},
"required": ["invoice_number", "vendor", "date", "total", "currency", "payment_days"]
}
}
}
}
EOF
curl -s http://127.0.0.1:9931/v1/chat/completions -H 'Content-Type: application/json' -d @/tmp/extract.json | jq '{answer: .choices[0].message.content, tok_s: .timings.predicted_per_second}'Compare each field with the input. The correct values are INV-2291, Northwind Supplies, the date, 1480, EUR and 30. Watch the amount most closely, because 1,480.00 uses a thousands separator that a small model can misread. If tok_s prints null, your build did not attach timing data to this response, so use the llama-bench figures instead.
To test it properly, run 20 or 30 real documents from your own work through the same request and count the wrong fields. That error rate on your data is the number that decides whether a 230M model is enough for the job.
A reasoning task inside the card's warning
Now ask for several steps of arithmetic, which is close to the "advanced math" the card warns about. The question has one correct answer, so you can judge the output without opinion.
cat > /tmp/reason.json <<'EOF'
{
"messages": [
{"role": "user", "content": "A train leaves at 09:40. The trip takes 2 hours and 35 minutes. The train is then 25 minutes late. A shop has 3 boxes of 17 apples. It sells 8 apples and then buys 2 more boxes. Answer two questions: at what time does the train arrive, and how many apples does the shop have now? Show your steps."}
],
"max_tokens": 300
}
EOF
curl -s http://127.0.0.1:9931/v1/chat/completions -H 'Content-Type: application/json' -d @/tmp/reason.json | jq -r '.choices[0].message.content'The correct answers are 12:40 and 77 apples. Check the final answers and each step. Run the request several times, and change the numbers too, because one correct answer can be luck. The card says the model is not recommended for this kind of work. If your runs show wrong steps or wrong totals, you are seeing the documented limit, and no sampling setting fixes it. For multi-step work, use a larger model. Liquid AI's larger option is covered in running LFM2.5-24B-A2B on a VPS.
Keep it running with systemd
A server started by hand stops when your SSH session ends. Write a unit file at /etc/systemd/system/lfm-230m.service:
[Unit]
Description=LFM2.5-230M on llama-server
After=network-online.target
Wants=network-online.target
[Service]
ExecStart=/opt/llama/llama-b12189/llama-server -m /opt/models/LFM2.5-230M-Q4_K_M.gguf -c 8192 -np 1 -t 2 --host 127.0.0.1 --port 9931 --temp 0.1 --top-k 50 --repeat-penalty 1.05
DynamicUser=yes
Restart=on-failure
[Install]
WantedBy=multi-user.targetsudo systemctl daemon-reload
sudo systemctl enable --now lfm-230m
systemctl status lfm-230m --no-pager
curl -s http://127.0.0.1:9931/healthDynamicUser=yes runs the server as a temporary unprivileged user, so the server process cannot write to the rest of the system. The files under /opt are readable by every user, so the server can still load them. If the unit fails, journalctl -u lfm-230m -n 50 shows the error. To upgrade llama.cpp, extract the new tag next to the old one, change the path in ExecStart, and restart. The old folder is your rollback.
Can Ollama pull LFM2.5-230M directly?
Yes, through Hugging Face, and you do not need a Modelfile. As of 7 November 2026, Ollama's official library has no lfm2.5-230m entry, and the LiquidAI namespace on ollama.com lists only lfm2.5-1.2b-instruct, lfm2.5-350m and lfm2.5-2.6b. Hugging Face documents a direct route instead, and the Liquid AI card gives this command:
ollama --version
ollama run hf.co/LiquidAI/LFM2.5-230M-GGUF:Q4_K_MThe tag after the colon picks the quant file, and Hugging Face says the tag is case-insensitive. Ollama picks the chat template from the metadata inside the GGUF file. It does not read the card's sampling advice, so set it in the session with /set parameter temperature 0.1, and set top_k and repeat_penalty the same way. The newest Ollama release on 7 November 2026 was v0.40.15, and its notes do not mention LFM2. Note the version that ollama --version prints if you report a problem. A community upload named oamazonasgabriel/lfm2.5-230m also exists. It does not come from Liquid AI, so the hf.co route is the safer choice. For the trade-offs between the two runtimes, read Ollama vs llama.cpp.
What the licence allows
The model uses the LFM Open License v1.0, tagged lfm1.0 on Hugging Face. It reads like Apache 2.0 with one large addition: a revenue threshold. The licence defines the threshold as "annual revenue of 10 million United States dollars ($10,000,000) or more". Commercial use is licensed only while you, or your legal entity, stay under that threshold. Commercial use by an entity above it is not licensed under this agreement. Qualified non-profit organisations are exempt from the threshold for non-commercial or research use.
If you redistribute the model or a fine-tuned version, you must include a copy of the licence, mark the files you changed, keep the existing notices, and carry over the attribution text from any NOTICE file. The licence ends automatically if you break any of its terms. This is a summary, not legal advice. Read the full text before you build a product on it.
Failure modes, with the messages you will see
The binary will not start. An error such as error while loading shared libraries names the file the loader could not find. Run ldd /opt/llama/llama-b12189/llama-server | grep 'not found' to list every missing library. If the missing files are the build's own libraries, you moved the binary out of its folder.
The server answers 503. /health returns 503 while the model is still loading. A 503 that does not go away means the load failed, so read the server's output or journalctl -u lfm-230m.
Answers ramble or repeat themselves. The server is probably running with its default sampling. Check that the startup command, or the ExecStart line, contains --temp 0.1 --top-k 50 --repeat-penalty 1.05.
Memory is much higher than you expect. Check the -c value. Without it the server takes the context size from the GGUF metadata, which is 128000, and reserves memory for a context the model card does not promise.
Connection refused from another machine. That is the --host 127.0.0.1 flag doing its job. Reach the server over an SSH tunnel, or put an authenticated reverse proxy in front of it. Do not bind it to 0.0.0.0 on a public VPS.
FAQ
How much RAM does LFM2.5-230M need?
The Q4_K_M file is about 153 MB and the BF16 file is about 462 MB, so the weights fit on any VPS with 1 GB of RAM. The total depends on the context size you set with -c. Start the server with -c 8192, then read its resident memory with ps -C llama-server -o rss= to get the real figure for your build.
Is LFM2.5-230M the same as LFM2.5-Encoder-230M?
No. LFM2.5-230M is a text generation model for extraction and light agent pipelines. LFM2.5-Encoder-230M is a bidirectional masked language model that Liquid AI offers as a base to fine-tune into embedding or retrieval models. It is not a chat model. Download LiquidAI/LFM2.5-230M-GGUF for chat and extraction.
What sampling settings should I use for LFM2.5-230M?
The model card recommends temperature 0.1, top_k 50 and repetition penalty 1.05. In llama.cpp the flags are --temp 0.1 --top-k 50 --repeat-penalty 1.05. Set them yourself, because llama-server defaults to temperature 0.80, top-k 40 and no repeat penalty.
Can LFM2.5-230M do math or write code?
The model card says it is not recommended for reasoning-heavy workloads such as advanced math, code generation or creative writing. Use it for extraction and simple tool calls, and check its answers on your own data. For multi-step reasoning or code, run a larger model.
Can I use LFM2.5-230M commercially?
Yes, while you or your legal entity have annual revenue under 10 million US dollars. The LFM Open License v1.0 does not license commercial use above that threshold. Redistribution needs a copy of the licence and notices of your changes. Read the licence file on the model's Hugging Face page before you ship a product.