Self-hosted speech to text and TTS on a VPS
Run Whisper transcription and Piper text to speech on your own VPS. What it costs in CPU time and disk, and how to expose an OpenAI compatible endpoint.
#self-hosted-ai
Filtering by topic #self-hosted-ai · clear
Which AI models can you self-host?
Pick a model by the RAM you actually have. Sizing arithmetic for 4 GB, 16 GB and 64 GB VPS plans, honest CPU token rates, and the hidden cost of context.
KV cache vs prompt cache explained
The KV cache is RAM your server pays for on every request. Prompt caching is a bill someone else discounts. Only one can break a model load.
Ollama context length: setting num_ctx
Ollama truncates long prompts at a small default context window. Set num_ctx per request or server wide, and size the KV cache RAM before you raise it.
Keep an Ollama model loaded in memory
Ollama unloads your model after 5 minutes idle, so the next request pays the full load time again. Set keep_alive so it sticks, even after a reboot.
Why your self-hosted LLM stalls at 5 users
One user was fine, five crawl. How batching, KV cache limits, prefill and queue depth decide how many people your LLM server can serve at once.
Use Ollama with your coding agent
Point a coding agent at a model you host: the base URL, the dummy key, the context length that breaks everything, and the jobs a local model wins.
GPU VPS vs API tokens: the break-even point
Renting a GPU by the hour beats per-token API billing only above a certain volume. Here is the formula and the monthly token count where it flips.
Self-host LiteLLM as your LLM gateway
Run one OpenAI-compatible endpoint in front of every provider you use: LiteLLM on a VPS with virtual keys, per-key budgets, fallbacks, and pinned images.
Run Qwen 3.8 27B on a VPS With Ollama
There is no Qwen 3.8 on Ollama yet. Here is the arithmetic for running the 27B tag that does exist on a CPU-only VPS, and what fits in 8 to 64 GB.
Open WebUI alternatives for a VPS
Open WebUI, LibreChat, Hollama and OrionChat compared on a VPS with a public IP: RAM left for the model, logins, remote Ollama, and upkeep.
Ollama vs llama.cpp on a VPS
llama.cpp is the engine, Ollama is the layer on top. Which one to run on a CPU-only VPS, how quantisation choice changes RAM, and when neither fits.