Ollama KV cache quantization: q8_0 vs f16
Raised num_ctx and Ollama no longer fits? The KV cache grew, not the weights. Set OLLAMA_KV_CACHE_TYPE with flash attention and watch the change in ollama ps.
#ram
Filtering by topic #ram · clear
How big an LLM fits in your server RAM
Size an LLM for a CPU server without guessing: weights in bytes, KV cache growth with context, runtime overhead, and why fitting is not the same as usable.
How much RAM does a coding agent VPS need?
One always-on coding agent runs on 4 GB and 2 vCPU. The builds and language servers it launches are what fill the box, and what makes it hang.