Why your self-hosted LLM stalls at 5 users
One user was fine, five crawl. How batching, KV cache limits, prefill and queue depth decide how many people your LLM server can serve at once.
#vllm
Filtering by topic #vllm · clear
Measure tokens per second on a local LLM
A rented GPU only beats per-token billing above a throughput floor. Measure tokens per second properly with a concurrency sweep, then decide.
GPU VPS vs API tokens: the break-even point
Renting a GPU by the hour beats per-token API billing only above a certain volume. Here is the formula and the monthly token count where it flips.
Ollama vs vLLM: which LLM server to run
Ollama is a convenience layer for one user, on CPU if needed. vLLM is a throughput engine for GPUs. Pick by workload, with the real commands for both.