SSD Nodes Learn Hosting plans →
Guides Matt ConnorBy Matt Connor

Run Laya on a VPS: pip, Ollaya or Ollama?

Host the Laya decision model on a CPU VPS with the laya 0.3.26 package, keep its server off the public internet, and see how Ollama and Ollaya run it.

Which runtime actually runs the Laya decision model?

To run the Laya decision model on a VPS, this guide installs the laya Python package in a virtual environment and starts its own server, laya-serve. The guide pins version 0.3.26, published on PyPI on 2026-10-03. As of 2026-11-21 the newest release is 0.5.2, published 2026-11-21. The steps and defaults below were written for 0.3.26, so keep the pin unless you re-check them against a newer release. Ollaya, an independent runtime, also lists Laya, and Ollama's own library ships an official laya model too.

Laya is a decision model from Convai Innovations, released under the Apache-2.0 license. The model card is convaiinnovations/laya on Hugging Face and the code lives at NandhaKishorM/laya on GitHub. A decision model does not write text. You send it a piece of text, called the state, plus a set of typed questions: yes or no, a choice between options, or a score. It returns a probability for every option in one forward pass. That is what you want for ticket routing, tagging and moderation, where the output is a label and a confidence, not a paragraph. Laya speaks the same HTTP contract as TypeSafe's hosted Jev API, POST /v1/systemone, so most of the guide to running a Jev-style decision model on a VPS applies here too.

These are the three runtimes people search for:

  • laya-serve from laya 0.3.26, installed from PyPI as laya[serve]. This is the reference server, from the same author as the model. It routes each request to the right Laya checkpoint. The rest of this guide uses it. Do not install the separate laya-serve project from PyPI. It stops at 0.2.1 and is marked "Deprecated / archived", with a pointer to laya[serve] instead.
  • Ollaya v0.9.0, released 2026-10-02 at ollaya-dev/ollaya by Mert Cobanov. It is an independent project that describes itself as Ollama for decision models. Its README lists laya:en and laya:multilingual, served on port 11435 behind a TypeSafe-compatible API. As of 2026-11-10 the newest release is v0.18.0, published 2026-10-25, and the README also lists laya:typed-decisions.
  • Ollama 0.40.0 or later. Ollama added decision models in version 0.35, announced in a blog post dated 2026-09-29, and serves them on /v1/systemone. As of 2026-11-10 its decision category lists fourteen models: autotrust-jev, pplx-decider-flash, d3, d1, d1-omni, pplx-decider, nimble-v3, kev, tev1, strands-decider, clef, nimble, clef-flash and laya. The laya page says it requires Ollama 0.40.0 or later. As of 2026-11-21 the newest stable Ollama release is v0.40.17, marked "Latest" with tag v0.40.17, released 2026-11-10.

Ollama's model library describes the official laya model as "a 421M decision model from Convai Innovations, fine-tuned from ModernBERT-large". The community upload s1gnature/laya is tagged as an embedding model, which returns vectors from /api/embed, so pull the official one for decision questions. Ollama's build is narrower than laya-serve. The default laya:latest tag has two runner variants. For llama.cpp it is an 844MB BF16 GGUF of the English checkpoint, and the 846MB entry is its MLX variant. The BF16 tags list an 8K context window, while the English MLX tag lists 512 tokens. Ollama's System One API docs say the complete input must fit the loaded context window and is never truncated, while the laya model page says each question gets a 512-token budget and longer states are truncated. Either way, shorten oversized tickets before submission. The tag list also has three MLX builds, laya:421m-english-mlx-fp16, laya:322m-multilingual-mlx-fp16 and laya:421m-typed-decisions-mlx-fp16, plus three BF16 builds, laya:421m-english-bf16, laya:322m-multilingual-bf16 and laya:421m-typed-decisions-bf16. Ollama's release notes describe its MLX runner on Apple silicon, so on a CPU VPS plan on the BF16 builds, starting with the default. The model page text still says English only even though a multilingual BF16 tag is listed, so test that tag on your own non-English text before you rely on it. The plain answer to "ollama laya" is yes, for English text, with ollama pull laya. Use laya-serve or Ollaya when you need the multilingual checkpoint routed for you.

What Convai claims, and what you should measure yourself

Every number in this section is Convai's own published figure. None of them was measured for this guide. Treat them as vendor claims until your own box confirms them.

ChartLaya checkpoints, as published by Convai
The data behind this chart
[
  {
    "label": "laya (English)",
    "params_millions": 421,
    "context_tokens": 512,
    "download_mb": 808
  },
  {
    "label": "laya-multilingual",
    "params_millions": 322,
    "context_tokens": "1,024",
    "download_mb": 647
  }
]

According to Convai, the English checkpoint has 421 million parameters, a context of 512 tokens, and a download of about 808 MB. The multilingual checkpoint has 322 million parameters, a context of 1,024 tokens, and a download of about 647 MB. A third checkpoint, laya-typed-decisions, is built on the same base model as the English one and targets domain-specific decision tasks.

Convai also claims better calibration than Jev. ECE (expected calibration error) measures the gap between the confidence a model states and how often it is actually right. Lower is better.

ChartCalibration error (ECE), Convai's published figures, lower is better
The data behind this chart
[
  {
    "label": "Laya",
    "ece": 0.081
  },
  {
    "label": "Jev",
    "ece": 0.246
  }
]

The model card gives Laya an ECE of 0.081 against 0.246 for Jev. The model card also says Laya answers a single question six to eight times faster than Jev. Those speed figures come from GPU runs, and a CPU VPS will be slower. Do not repeat the vendor's calibration numbers as fact in your own planning. The Jev guide above includes a calibration check you can run against your own labelled data, and it works unchanged against laya-serve because the API is the same.

Install laya-serve 0.3.26 in a virtual environment

Ubuntu 24.04 marks the system Python as externally managed, so a bare pip install fails with error: externally-managed-environment. A virtual environment avoids that and keeps Laya's dependencies away from the system packages. If you are deciding between a venv, pipx and uv for server tools, the comparison of venv, pipx and uv on a server covers the trade-offs. This guide uses a plain venv in /opt/laya, owned by root, and runs the server as its own system user.

sudo apt update && sudo apt install -y python3-venv
sudo useradd --system --create-home --home-dir /var/lib/laya --shell /usr/sbin/nologin laya
sudo python3 -m venv /opt/laya
sudo /opt/laya/bin/pip install --upgrade pip
sudo /opt/laya/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu
sudo /opt/laya/bin/pip install "laya[serve]==0.3.26"

The torch line comes first for a reason. Laya runs on PyTorch, and on Linux the default torch wheel from PyPI pulls in the NVIDIA CUDA libraries. That is several gigabytes of disk a CPU VPS never uses. Installing the CPU build from PyTorch's own index first means pip finds torch already present when it resolves Laya. The [serve] extra adds FastAPI and uvicorn, which laya-serve needs.

Check both versions:

/opt/laya/bin/pip show laya | head -2
/opt/laya/bin/python -c "import torch; print(torch.__version__)"

The first command should print Version: 0.3.26. The second should print a version ending in +cpu. If it does not end in +cpu, pip replaced your CPU build with the default wheel, usually because Laya asked for a torch version the CPU index did not have.

What does laya-serve bind to by default?

The PyPI page says laya-serve binds 0.0.0.0:8000. That means every network interface, including the public one. Do not take the docs on trust. Read the line in the release you installed:

grep -n 'LAYA_HOST' /opt/laya/lib/python3*/site-packages/laya/serve.py

In the published source the default is os.environ.get("LAYA_HOST", "0.0.0.0"). If your output shows that line, the docs are right for your version. On a VPS with a public IP and no firewall rule in the way, anyone who finds port 8000 can send requests and spend your CPU. Authentication is also off unless you set LAYA_API_KEY, so the default is an open endpoint.

Start the server in the foreground with loopback set explicitly:

sudo -u laya -H env LAYA_HOST=127.0.0.1 LAYA_PORT=8000 LAYA_DEVICE=cpu /opt/laya/bin/laya-serve

By default the server preloads every checkpoint at startup, so the first start downloads and loads all three before it accepts connections. In a second SSH session, check what is actually listening:

ss -ltn 'sport = :8000'

If nothing is listed yet, the checkpoints are still loading, so wait and run it again. The Local Address column should read 127.0.0.1:8000. If it reads 0.0.0.0:8000, the variable did not reach the process. The usual cause is that sudo dropped it, which is why the command above passes it through env. The -H flag sets HOME to /var/lib/laya, so the checkpoint download lands in that user's cache and not in yours.

Send the first /v1/systemone request

Save the README's example question to a file so you can send it again:

cat > q.json <<'EOF'
{"state": {"body": "billed twice, refund please or we cancel"},
 "questions": {"dept": {"type": "choice", "instructions": "which team?",
   "criteria": {"billing": "refunds", "tech": "bugs"}}}}
EOF
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d @q.json | python3 -m json.tool

The response holds an answers object keyed by your question name, dept, with a probability for each option. For this text, billing should get the higher probability. With default settings the slow part came before this request. At startup the server downloaded every checkpoint, about 808 MB for English and 647 MB for multilingual plus the typed-decisions checkpoint, and loaded them into memory. If you set LAYA_PRELOAD=0, that download and load move to the first request that needs each checkpoint. Either way, requests after the first show the real speed.

The server also has a health route, which reports checkpoint status and the device in use:

curl -s localhost:8000/health | python3 -m json.tool

The device should say cpu. With the default preload, all three checkpoints should show as loaded. If one does not, LAYA_MODELS or LAYA_PRELOAD=0 left it out, and it loads on first use.

Measure RAM and latency on your own VPS

Published latency comes from a GPU. Your number depends on your CPU model and your vCPU count, so take it yourself after the first warm-up request:

for i in 1 2 3 4 5; do
  curl -s -o /dev/null -w '%{time_total}\n' localhost:8000/v1/systemone \
    -H 'content-type: application/json' -d @q.json
done
ps -o rss=,cmd= -u laya

The loop prints the total time of each request in seconds. The ps line prints resident memory in kilobytes for each process run by the laya user. Note both figures with the plan size you tested on.

Several settings change memory use. With server defaults, LAYA_PRELOAD is 1 and an empty LAYA_MODELS means every checkpoint, so all three load at startup and the router raises the LAYA_MAX_LOADED cap (2 by default) to fit them. The ps figure you just took therefore already covers the full set. It does not grow when the first Spanish ticket arrives. To keep fewer checkpoints in memory, set LAYA_MODELS to the ones you need, for example LAYA_MODELS=english, or set LAYA_PRELOAD=0 to load lazily. On-demand loads evict the least-recently-used checkpoints until the cap holds, including preloaded checkpoints. Preloading raises the cap to fit its checkpoints, so LAYA_MAX_LOADED=1 on its own does not get you down to one checkpoint. If you need one at a time, combine it with LAYA_PRELOAD=0 or a one-name LAYA_MODELS. With LAYA_MODELS=english and LAYA_MAX_LOADED=1, loading multilingual evicts the preloaded English checkpoint. Expect a reload each time routing switches checkpoints, because the router sends non-English text to the multilingual one. LAYA_IDLE_UNLOAD_SECONDS unloads checkpoints that have been idle, and the next request loads them again. If Laya shares the box with other services, LAYA_THREADS caps the CPU threads it uses, so it does not take every core.

Add bearer auth with LAYA_API_KEY and run it under systemd

When LAYA_API_KEY is set, every decision request needs an Authorization: Bearer <key> header. Put the settings in a root-only file, then write a systemd unit that reads it.

sudo install -m 600 /dev/null /etc/laya.env
printf 'LAYA_HOST=127.0.0.1\nLAYA_PORT=8000\nLAYA_DEVICE=cpu\nLAYA_API_KEY=%s\n' "$(openssl rand -hex 32)" | sudo tee /etc/laya.env > /dev/null

Create /etc/systemd/system/laya.service:

[Unit]
Description=Laya decision model server
After=network-online.target
Wants=network-online.target

[Service]
User=laya
Group=laya
EnvironmentFile=/etc/laya.env
Environment=HOME=/var/lib/laya
ExecStart=/opt/laya/bin/laya-serve
Restart=on-failure

[Install]
WantedBy=multi-user.target

Stop the foreground server with Ctrl+C, then start the service:

sudo systemctl daemon-reload
sudo systemctl enable --now laya
sudo systemctl status laya --no-pager

Now test both sides of the key:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d @q.json
KEY=$(sudo sed -n 's/^LAYA_API_KEY=//p' /etc/laya.env)
curl -s localhost:8000/v1/systemone -H 'content-type: application/json' \
  -H "Authorization: Bearer $KEY" -d @q.json | python3 -m json.tool

The first call should return HTTP 401 with {"detail":"invalid or missing bearer token"}. The second should return the answers as before. Run ss -ltn 'sport = :8000' again and confirm the service still listens on 127.0.0.1. Once a key is set, /health gives an unauthenticated caller only {"status": "ok"}, so send the bearer header when you want the checkpoint and device details.

A key is not encryption. Over plain HTTP the bearer token crosses the network in cleartext, so a remote client should reach Laya through a TLS reverse proxy, with Laya itself staying on loopback. The pattern is the same one used for Ollama, and securing a self-hosted model API endpoint walks through the proxy, TLS and firewall steps.

Does a Jev client work after changing baseUrl?

The Laya README says an existing Jev client only needs its baseUrl pointed at laya-serve, and that the response schema is identical. One real request settles it for your client. If your client is the TypeSafe SDK, it reads TYPESAFE_BASE_URL and TYPESAFE_API_KEY. Set the first to http://127.0.0.1:8000 and the second to your LAYA_API_KEY, then send one question your code already sends to Jev. A pass means your code parses the answer exactly as before. A 401 means the key or the header is wrong.

The README also lists the porting differences, and these are where a working Jev request can break:

  • A question's options share one token budget per checkpoint, 192 tokens on laya and 256 on the other two. About 20 options with a short description each fill that budget, so treat 20 as a quality guideline, not a limit. Past that, every option description is trimmed to fit, and a request that no longer fits at all is rejected with 422. The server's hard cap is 100 choice options per question.
  • Every score level needs a description. A level with no description is not accepted.
  • Confidence is calibrated differently, so a threshold you tuned on Jev's confidence values will not mean the same thing on Laya.

The last point matters most in production. Re-tune any threshold before you move live traffic, using the calibration check linked above.

Running Laya through Ollaya instead

Ollaya ships as a single binary with Ollama-style commands: pull, run, list, ps and serve. Its README gives a pipe-to-shell installer. Download it and read it before you run it:

curl -fsSL https://ollaya.dev/install.sh -o install-ollaya.sh
less install-ollaya.sh
sh install-ollaya.sh
ollaya pull laya:en

If the installer did not start a background service, run ollaya serve. Per its documentation it listens on 127.0.0.1:11435 by default, and OLLAYA_HOST changes that. So, unlike laya-serve, it starts on loopback. Requests use the same /v1/systemone body, plus a model field set to laya:en or laya:multilingual. Port 11435 sits next to Ollama's 11434, so both can run on one box. How Ollama's API uses port 11434 explains the Ollama side if you run both. The procedure in this guide was written for laya-serve. The Ollaya commands here follow its README for v0.9.0.

Pick laya-serve if Laya is the only model you need and you want the reference server pinned through pip. Pick Ollaya if you want several decision model families behind one daemon. If you are already on Ollama for its own decision models, Laya now runs there too, as the next section shows, and the ticket triage setup with Ollama on a VPS shows the same kind of routing job end to end.

Running Laya through Ollama

Ollama's official laya model needs Ollama 0.40.0 or later. Check your version, pull the model, and send the same question with a model field added:

ollama -v
ollama pull laya
cat > q-ollama.json <<'EOF'
{"model": "laya",
 "state": {"body": "billed twice, refund please or we cancel"},
 "questions": {"dept": {"type": "choice", "instructions": "which team?",
   "criteria": {"billing": "refunds", "tech": "bugs"}}}}
EOF
curl -s localhost:11434/v1/systemone -H 'content-type: application/json' -d @q-ollama.json | python3 -m json.tool

The first line should print a version of 0.40.0 or higher. As of 2026-11-11 the model page includes API, Python and JavaScript examples for decision requests, using Ollama's own Python and JavaScript libraries through ollama.systemone; its notes no longer list a decision-support limitation for those libraries. ollama-python v0.6.3, released 2026-09-29, added System One API support, with a systemone method that posts to /v1/systemone. For JavaScript, System One support is in ollama-js v0.6.4, which is still marked as a pre-release. The default tag is the English checkpoint; its BF16 build lists an 8K context window and its MLX build lists 512 tokens. Ollama's API docs say the complete input must fit the loaded context window and is never truncated, but the model page says longer states are truncated, so shorten oversized tickets before submission. Time it with the same curl -w loop as above, pointed at port 11434, and re-tune your thresholds on your own data before you move traffic. Ollama's port needs the same care as laya-serve, and the endpoint guide linked above applies unchanged.

FAQ

Can Ollama run the Laya decision model?

Yes, as of 2026-11-10. Ollama's library has an official laya model that requires Ollama 0.40.0 or later and answers on /v1/systemone. The default laya:latest tag is the English checkpoint; its BF16 build lists an 8K context window and its MLX build lists 512 tokens. The community upload s1gnature/laya is an embedding model that returns vectors. Ollama's tag list includes laya:322m-multilingual-bf16, but the model page still describes Laya as English only, so test that tag first. If you need the multilingual checkpoint routed for you on a CPU VPS, use laya-serve from PyPI or the independent Ollaya runtime, which lists laya:en and laya:multilingual.

Is laya-serve safe to expose on port 8000?

Not with its defaults. The server reads LAYA_HOST and falls back to 0.0.0.0, which listens on every interface, and it has no authentication unless LAYA_API_KEY is set. Set LAYA_HOST=127.0.0.1 and an API key, confirm the bind with ss -ltn 'sport = :8000', and put a TLS reverse proxy in front if remote clients need it.

How much RAM does Laya need on a CPU VPS?

Measure it on your own plan. Send one warm-up request, then run ps -o rss=,cmd= -u laya to see resident memory in kilobytes. With server defaults all three checkpoints load at startup, because LAYA_PRELOAD defaults to 1 and an empty LAYA_MODELS means every checkpoint. To keep memory lower, set LAYA_MODELS to only the checkpoints you need or set LAYA_PRELOAD=0. Then add LAYA_MAX_LOADED=1 or LAYA_IDLE_UNLOAD_SECONDS if you want fewer resident at once. Preloading raises the cap to fit its checkpoints, so LAYA_MAX_LOADED=1 on its own still loads all three with the default preload. With just one checkpoint preloaded and the cap at one, loading another evicts the preloaded checkpoint.

Why is the first Laya request so slow?

With default settings the slow part is the first start, not the first request. The server downloads every checkpoint, about 808 MB for English and 647 MB for multilingual according to Convai, plus the typed-decisions checkpoint, and loads them before it accepts connections. With LAYA_PRELOAD=0 that work moves to the first request that needs each checkpoint. Later requests skip both steps. Time a few requests with curl -w '%{time_total}' after the first one to see your real latency.

Do I need a GPU to run Laya?

No. Set LAYA_DEVICE=cpu and Laya runs on an ordinary VPS. Convai's speed figures come from GPU runs, so expect higher latency on CPU, and measure it before you decide whether it is fast enough for your traffic.

Was this article useful?

#laya#decision-models#ollama#self-hosted-ai#text-classification