Self-host Qwen3.8-Flash-Next: what it takes
Run Qwen3.8 Flash Next on your own hardware: how much RAM the GGUF files need, where Strata fits, when you need vLLM, and what the licence restricts.
Filtering by topic #llama-cpp · clear
Run Qwen3.8 Flash Next on your own hardware: how much RAM the GGUF files need, where Strata fits, when you need vLLM, and what the licence restricts.
llama.cpp now cuts version tags beside its bNNNN build tags. Pin one, record it with the GGUF and quant, and treat every upgrade as a drill you can undo.
Build llama.cpp from a pinned tag, serve GGUF models with OpenAI API compatibility, bind it to localhost, and run it under systemd with memory limits.
Serve Liquid AI's 230M model with a pinned llama-server build on a 1 or 2 vCPU box, then test where it extracts clean JSON and where its reasoning fails.
llama.cpp is the engine, Ollama is the layer on top. Which one to run on a CPU only VPS, how quantisation choice changes RAM, and when neither fits.
Flint has 40B parameters but only 2.5B active per token. Size the RAM from config.json, cap the context, pin the build, and see what runs on a CPU VPS today.
Run LFM2.5-24B-A2B on a CPU-only VPS with llama-server: size RAM from the quant table, pin llama.cpp, set the card's sampling and keep the API on localhost.