Import a GGUF model into Ollama
Run a .gguf from Hugging Face or from a local file under Ollama, and fix the chat template mismatch that makes an imported model reply in garbage.
#gguf
Filtering by topic #gguf · clear
Ollama quantization: Q4 vs Q8 vs fp16
Pick an Ollama quantization with arithmetic instead of guesswork: what q4_K_M, q8_0 and fp16 cost in RAM, and where the quality actually drops.
Ollama vs llama.cpp on a VPS
llama.cpp is the engine, Ollama is the layer on top. Which one to run on a CPU-only VPS, how quantisation choice changes RAM, and when neither fits.