Keep an Ollama model loaded in memory
Ollama unloads your model after 5 minutes idle, so the next request pays the full load time again. Set keep_alive so it sticks, even after a reboot.
#self-hosted-ai
Filtering by topic #self-hosted-ai · clear
Ollama context length: setting num_ctx
Ollama truncates long prompts at a small default context window. Set num_ctx per request or server wide, and size the KV cache RAM before you raise it.
Use Ollama with your coding agent
Point a coding agent at a model you host: the base URL, the dummy key, the context length that breaks everything, and the jobs a local model wins.
Why your self-hosted LLM stalls at 5 users
One user was fine, five crawl. How batching, KV cache limits, prefill and queue depth decide how many people your LLM server can serve at once.
GPU VPS vs API tokens: the break-even point
Renting a GPU by the hour beats per-token API billing only above a certain volume. Here is the formula and the monthly token count where it flips.
Self-host LiteLLM as your LLM gateway
Run one OpenAI-compatible endpoint in front of every provider you use: LiteLLM on a VPS with virtual keys, per-key budgets, fallbacks, and pinned images.
Run Qwen 3.8 27B on a VPS With Ollama
There is no Qwen 3.8 on Ollama yet. Here is the arithmetic for running the 27B tag that does exist on a CPU-only VPS, and what fits in 8 to 64 GB.
Open WebUI alternatives for a VPS
Open WebUI, LibreChat, Hollama and OrionChat compared on a VPS with a public IP: RAM left for the model, logins, remote Ollama, and upkeep.
Ollama vs llama.cpp on a VPS
llama.cpp is the engine, Ollama is the layer on top. Which one to run on a CPU-only VPS, how quantisation choice changes RAM, and when neither fits.