Import a GGUF model into Ollama
Run a .gguf from Hugging Face or from a local file under Ollama, and fix the chat template mismatch that makes an imported model reply in garbage.
#local-llm
Filtering by topic #local-llm · clear
Pin llama.cpp releases on your server
llama.cpp now cuts v0.x tags beside its bNNNN build tags. Pin one, record it with the GGUF and quant, and treat every upgrade as a drill you can undo.
Run Meta Muse Glimmer 30B on a VPS
Muse Glimmer tags run from 17GB to 59GB. Work out the RAM and disk a rented Linux VPS needs before you pull, and what CPU only inference costs.
Ollama pull vs run, and where models live
ollama pull downloads a model and stops. ollama run downloads it, then opens a chat. Where the files land, why they fill a VPS root disk, how to move them.
Ollama num_predict: cap the output length
num_predict caps how many tokens Ollama will write. The three places to set it, which one wins, and how to read done_reason in the response.
Run Nemotron 3.5 Lightning on a VPS
Run NVIDIA Nemotron 3.5 Lightning with Ollama on your own server: the exact tag to pull, how much RAM it needs, and if CPU-only is fast enough.
Ollama quantization: Q4 vs Q8 vs fp16
Pick an Ollama quantization with arithmetic instead of guesswork: what q4_K_M, q8_0 and fp16 cost in RAM, and where the quality actually drops.
Use Ollama with your coding agent
Point a coding agent at a model you host: the base URL, the dummy key, the context length that breaks everything, and the jobs a local model wins.
Ollama vs llama.cpp on a VPS
llama.cpp is the engine, Ollama is the layer on top. Which one to run on a CPU-only VPS, how quantisation choice changes RAM, and when neither fits.