Ollama quantization: Q4 vs Q8 vs fp16
Pick an Ollama quantization with arithmetic instead of guesswork: what q4_K_M, q8_0 and fp16 cost in RAM, and where the quality actually drops.
#gguf
Filtering by topic #gguf · clear
Ollama vs llama.cpp on a VPS
llama.cpp is the engine, Ollama is the layer on top. Which one to run on a CPU-only VPS, how quantisation choice changes RAM, and when neither fits.