Fix Ollama's 'requires MLX support' error

What the "requires MLX support" error means

The Ollama error this model requires MLX support, but the MLX runtime is not available means the tag you asked for needs Ollama's MLX engine, and your Ollama server cannot start an MLX runtime. On a CPU-only VPS the fix is to pull a different tag of the same model, one that the llama.cpp engine runs, such as a q4_K_M or q8_0 tag. Upgrading Ollama alone does not remove the error, because the check has been in every release since v0.32.10.

The message appears right after the manifest step of a pull:

pulling manifest
this model requires MLX support, but the MLX runtime is not available

Ollama's source and the bug reports both write it with a capital MLX. If you search logs or GitHub issues with a case-sensitive search, type it that way. A lowercase search will miss it.

Why Ollama refuses the pull

Ollama has two engines. The llama.cpp engine runs GGUF files, the single-file weight format that llama.cpp reads. The MLX engine runs models stored as safetensors. Ollama's developer docs say it plainly: "The MLX engine enables running safetensor based models."

The same docs set the platform rules. They are a build guide for people who compile Ollama from source, so read them as requirements, not as a list of what each download ships. On macOS arm64, MLX is enabled by default. On Linux and Windows, MLX engine support is optional, and it requires CUDA 13+ and cuDNN 9+. CUDA is NVIDIA's GPU compute toolkit, and cuDNN (CUDA deep neural network library) is its neural-network add-on. A CPU VPS has no NVIDIA GPU, so it cannot meet either requirement, and the MLX runtime check fails.

Many readers first hit this with Ollama v0.40.0. Its first release candidate, v0.40.0-rc0, was tagged on 25 September 2026, and GitHub's release entry still shows that date. The final v0.40.0 tag came on 6 October 2026, after six more release candidates, and it includes commits made between 28 September and 6 October. Its notes say "Models run on MLX on Apple Silicon by default" and "MLX now has support for an embedding model: embeddinggemma-2." So EmbeddingGemma 2 arrived on Ollama as an MLX model. Issue #18825, "Failed to pull embeddinggemma-2:740m on linux", was filed against 0.40.0 on a Linux machine with an Intel CPU and GPU. A maintainer replied that the model was not yet available on Linux and Windows. Users then reported the same string on Windows and in Docker on TrueNAS Scale, for tags that included :270m, :740m-bf16 and several -text tags.

Does the error fire at pull or at run?

It fires at pull. In Ollama's source on main, read on 2026-10-17, the check sits in PullModel in server/images.go. That function fetches the manifest first. If the manifest needs MLX, it calls mlx.CheckInit(). When that call fails, PullModel returns the error as a plain errors.New value. All of this happens before any layer downloads, so a refused pull downloads no model weights and writes none to disk.

The run path does not use this string. When a model that is already on disk fails to load on the MLX runner, the scheduler in server/sched.go reports it through its generic server-start error handling. So this exact message tells you the failure happened during a pull. That includes the pull that ollama run starts by itself when the model is missing, which the guide to ollama pull vs run and where models are stored covers in detail.

Which tags does Ollama treat as MLX tags?

The decision is a small function, requiresMLX. If the manifest declares a runner, Ollama checks whether that runner is MLX, ignoring case. Manifests written before the runner field existed are judged by their layers: a manifest made of tensor layers needs MLX. When a name points at a manifest list, PullModel first picks the child manifest for the requested runner, then checks that child.

The check arrived in PR #17710, "mlx: avoid pulling MLX models when MLX is missing", merged on 12 August 2026. The PR states the reason: to avoid "users wasting time downloading a model they can't run". The first release that contains it is v0.32.10, published the same day and marked as a pre-release on GitHub. Every later tag inherits it, the whole v0.40.x line included, and it is still on main as of 2026-10-17.

Check a tag before you pull it

You can find out which engine a tag needs before you spend bandwidth on it. Work through these steps in order. The later ones are slower but more reliable.

Step 1: read the tags page

Open the model's tags page in a browser, for example https://ollama.com/library/embeddinggemma-2/tags. The MLX badge appears on tags whose names contain mlx, mxfp8 or nvfp4, but it does not mark every tag that needs MLX. Treat the badge as a hint only. Issue #18971 reported that embeddinggemma-2 tags without the badge, 740m and latest among them, were safetensors-only and failed on Linux and Windows. A commenter on the same issue said on 2026-10-16 that some MLX-only tags still lacked the badge. On 2026-10-17 the embeddinggemma-2 bf16 tags show no badge, yet a user reported this error for 740m-bf16 on 7 October. Check the manifest before you trust a missing badge.

Step 2: compare digests

The tags page shows a 12-character digest next to each tag. Two tags with the same digest point at the same files under two names. On 2026-10-17, the nemotron-3.5-lightning tags 30b-a3b-nvfp4, 30b-a3b-mlx and 30b-mlx all show digest d1e17d7dabac. So nvfp4 is an MLX tag here, even though its name does not say mlx. The GGUF-style tags have their own digests: 30b-a3b-q4_K_M is e7a64ff15fb1 and 30b-a3b-q8_0 is 9983b24ee511. Digests change when a tag is pushed again, so compare them on the day you pull.

Watch the model name too. The library entry called plain nemotron is Llama-3.1-Nemotron-70B, with 70b-instruct tags and no nvfp4 or mlx tags. The 30b-a3b tags belong to nemotron-3.5-lightning.

Step 3: read the manifest

The registry serves each tag's manifest as JSON, and the layer types in it are what requiresMLX looks at. These two commands count the layer types for an MLX tag and for a q8_0 tag of the same model:

curl -s https://registry.ollama.ai/v2/library/embeddinggemma-2/manifests/270m-nvfp4-text | grep -o 'application/vnd.ollama.image.[a-z]*' | sort | uniq -c
curl -s https://registry.ollama.ai/v2/library/embeddinggemma-2/manifests/270m-q8_0-text | grep -o 'application/vnd.ollama.image.[a-z]*' | sort | uniq -c

The first command prints a large count of application/vnd.ollama.image.tensor layers, plus some application/vnd.ollama.image.json layers. One layer per tensor is the safetensors layout, and a manifest built that way is an MLX manifest. The second command prints a single application/vnd.ollama.image.model layer. That one layer is the weights file the llama.cpp engine loads.

Some manifests also name their runner and format. Check for those keys like this:

curl -s https://registry.ollama.ai/v2/library/embeddinggemma-2/manifests/latest | grep -oE '"(runner|format)": *"[a-z]*"'

On 2026-10-17 the manifest for embeddinggemma-2 latest returned a runner of llamacpp and a format of gguf, and its layers name embeddinggemma-2:740m-q8_0 as their source. The MLX manifests read the same day had no runner key at all, and neither did the nemotron-3.5-lightning 30b-a3b-q4_K_M manifest. So an empty result from this grep proves nothing. The layer types from the first check decide.

Step 4: pull the GGUF tag and confirm it works

ollama --version
ollama pull embeddinggemma-2:270m-q8_0-text
ollama list
curl -s http://localhost:11434/api/embed -d '{"model": "embeddinggemma-2:270m-q8_0-text", "input": "hello"}'

The pull should end with success, and ollama list should show the tag. The last command should return JSON that holds an embeddings array of numbers. If the pull still prints the MLX error, the tag you named is an MLX tag, so go back to step 3. For batching inputs and storing the vectors, follow running EmbeddingGemma 2 on a CPU VPS.

Fix it for nemotron-3.5-lightning

The same method works for the larger model. Pick 30b-a3b-q4_K_M or 30b-a3b-q8_0 rather than an nvfp4, mxfp8 or mlx tag. The v0.40.5 release notes, published on 14 October 2026, say "Updated llama.cpp (adding Nemotron 3.5 support) and MLX". So upgrade the server to 0.40.5 or later before you pull a GGUF tag of this model.

ollama --version
ollama pull nemotron-3.5-lightning:30b-a3b-q4_K_M

Check your RAM first. On 2026-10-17 the tags page lists 30b-a3b-q4_K_M at 25GB, which is larger than the 19GB nvfp4 tag it replaces. The page lists the default tags (latest, 30b, 30b-a3b) with a size range of 19GB to 25GB, which suggests one name with more than one variant behind it. Naming the quantised tag yourself removes that guesswork. If you are not sure whether q4 or q8 suits your box, the comparison of Ollama quantization levels q4, q8 and fp16 covers the memory and quality trade.

Still the same error after upgrading Ollama?

That is the expected result, not a failed upgrade. The pull-time check has shipped in every release since v0.32.10. An upgrade only helps when the model you want also gets a non-MLX tag, and for embeddinggemma-2 no release note claims that.

Here is what the recent releases actually say. v0.40.4, released on 12 October 2026, lists "Updated llama.cpp with native decision model support", and it adds multimodal embedding support through llama-server. The llama.cpp bump in it is commit adc4aae9, titled "llama: update to b11521 and use native decision support". Neither the release notes nor that commit title names EmbeddingGemma. The decision-model work matters if you run a decision model such as Laya on a VPS. It does not change which embeddinggemma-2 tags are MLX tags. As of 2026-10-18, v0.40.6, released on 16 October 2026, is the latest stable release, and its notes have no MLX or embedding items. The v0.40.7-rc0 pre-release, published on 18 October 2026, adds MLX support for PPLX Decider v1.2 and has no embedding items. Issue #18825 is still open, with no linked pull request.

What did change is on the registry side. On 2026-10-18, embeddinggemma-2 latest resolves to llama.cpp layers taken from 740m-q8_0. That needs no upgrade, but it does need a new pull. Work through this list when the error will not go away:

• Name an explicit GGUF tag, such as :270m-q8_0-text or :740m-q8_0. An MLX tag such as :270m-nvfp4-text fails on every version since 0.32.10, because the tag itself is MLX.
• Pull again after a re-tag. Run ollama pull embeddinggemma-2 once more, so Ollama reads the current manifest for latest.
• Upgrade the server, not only the client. The check runs inside the server's PullModel, so the server version is the one that matters. In Docker, run docker exec -it ollama ollama --version to see the version inside the container.
• Run the manifest check from step 3. If the layers are still image.tensor layers, no Ollama version will pull that tag on a CPU-only machine.

Ollama's Linux docs say to update by running the install script again. For Docker, pull the ollama/ollama image again and recreate the container.

curl -fsSL https://ollama.com/install.sh | sh
docker pull ollama/ollama

If you installed by hand from the tarball instead, the same docs say to remove the old libraries with sudo rm -rf /usr/lib/ollama before you upgrade.

One user on issue #18971 reported getting past the error by installing libquadmath0 inside a Docker container. Nobody explained why, and nobody else confirmed it, so treat it as a single report, not a fix.

Other MLX errors that look similar

Not every MLX message is this one. Issue #17490 shows MLX not available: failed to load MLX dynamic library on Linux. That error came from importing a compressed-tensors safetensors model with --experimental, which sent the model to the MLX runner. It was first reported on Ollama 0.32.5 and closed as not planned on 2026-08-12. It is a create-time failure, not a pull-time one, so the tag checks above do not apply. If you hold your own weights, converting them to GGUF and following the steps to import a GGUF model into Ollama keeps the model on the llama.cpp engine. When a model has no GGUF tag at all, running llama.cpp directly instead of Ollama is the other route to try once a GGUF build of it exists.

FAQ

Why does Ollama say "this model requires MLX support" on my Linux VPS?

The tag you pulled is an MLX tag, and your Ollama server has no MLX runtime. Ollama's build docs say MLX is enabled by default only on macOS arm64. On Linux it is optional and requires CUDA 13+ and cuDNN 9+, which a CPU-only VPS does not have. Pull a GGUF-style tag of the same model, such as a q4_K_M or q8_0 tag, instead.

Does upgrading Ollama fix the MLX runtime error?

Not by itself. The pull-time check was added in PR #17710 and has been in every release since v0.32.10, and it is still in the source on main as of 2026-10-17. A tag whose manifest needs MLX fails on every one of those versions. The error goes away when you pull a non-MLX tag, or when the registry points the name you use at non-MLX layers.

How do I know if an Ollama tag needs MLX before I pull it?

Look for mlx, mxfp8 or nvfp4 in the tag name on the model's tags page, but do not trust a missing MLX badge, because some MLX-only tags lack it. Then compare digests, because a tag that shares a digest with an -mlx tag is the same files. For a firm answer, fetch the manifest from registry.ollama.ai and count its layer types. Many image.tensor layers mean MLX, and a single image.model layer means llama.cpp.

Can I install MLX on a CPU-only server to make the tag work?

Ollama's developer docs list CUDA 13+ and cuDNN 9+ as requirements for MLX engine support on Linux, and both need an NVIDIA GPU. A CPU-only server cannot meet them. The practical fix on that hardware is a GGUF-style tag, or importing a GGUF file of the model yourself.