Run Laya on a VPS: pip, Ollaya or Ollama?
Host the Laya decision model on a CPU VPS with the laya 0.3.26 package, keep its server off the public internet, and see how Ollama and Ollaya run it.
#self-hosted-ai
Filtering by topic #self-hosted-ai · clear
Run LFM2.5-230M locally on a CPU or small VPS
Serve Liquid AI's 230M model with a pinned llama-server build on a 1 or 2 vCPU box, then test where it extracts clean JSON and where its reasoning fails.
Ollama vs llama.cpp on a VPS
llama.cpp is the engine, Ollama is the layer on top. Which one to run on a CPU only VPS, how quantisation choice changes RAM, and when neither fits.
Self-hosted speech to text and TTS on a VPS
Run Whisper transcription and Piper text to speech on your own VPS. What it costs in CPU time and disk, and how to expose an OpenAI compatible endpoint.
Ollama context length: set num_ctx, size the RAM
Ollama truncates long prompts at a small default context window. Set num_ctx per request or OLLAMA_CONTEXT_LENGTH server wide, and size the KV cache RAM first.
Keep an Ollama model loaded in memory
Ollama unloads a model after 5 minutes idle, so the next request waits for a reload. Set OLLAMA_KEEP_ALIVE in a systemd override, then preload after reboot.
Run EmbeddingGemma 2 on a CPU-only VPS
Serve EmbeddingGemma 2 on a CPU VPS with Sentence Transformers, check whether the new Ollama GGUF tags run on your box, and store 256d vectors in pgvector.
Open WebUI alternatives for a VPS
Open WebUI, LibreChat, Hollama and OrionChat compared on a VPS with a public IP: RAM left for the model, logins, remote Ollama, and upkeep.
Run Qwen 3.8 27B on a VPS With Ollama
Qwen 3.8 27B is now on Ollama as qwen3.8:27b. Here is the RAM and speed arithmetic for running it on a CPU only VPS, and what fits in 8 to 64 GB.
Self-host LiteLLM as your LLM gateway
Run one OpenAI compatible endpoint in front of every provider you use: LiteLLM on a VPS with virtual keys, per key budgets, fallbacks, and pinned images.
Use Ollama with your coding agent
Point a coding agent at a model you host: the base URL, the dummy key, the context length that breaks everything, and the jobs a local model wins.
Why your self-hosted LLM stalls at 5 users
One user was fine, five crawl. How batching, KV cache limits, prefill and queue depth decide how many people your LLM server can serve at once.
Air-gapped AI server: what it takes
An air gap means no network path at all, and a rented VPS cannot give you one. Build egress-denied inference instead: default-deny outbound, staged weights.
GPU VPS vs API tokens: the break-even point
Renting a GPU by the hour beats per-token API billing only above a certain volume. Here is the formula and the monthly token count where it flips.
Which AI models can you self-host?
Pick a model by the RAM you actually have. Sizing arithmetic for 4 GB, 16 GB and 64 GB VPS plans, honest CPU token rates, and the hidden cost of context.
KV cache vs prompt cache explained
The KV cache is RAM your server pays for on every request. Prompt caching is a bill someone else discounts. Only one can break a model load.