Ollama:rocm Explained: AMD GPU Support Without the Guesswork

Start with the GPU architecture. Ollama can use ROCm when its bundled runtime has a compatible target for the AMD device, but a Radeon badge alone proves neither official support nor successful acceleration.

ROCm identifies each GPU by an architecture target such as gfx1100, and the loaded userspace libraries must contain code for that target. Ollama publishes a changing set of supported AMD families in its GPU compatibility documentation, while AMD maintains separate compatibility matrices for operating systems, drivers, and accelerators. Those lists overlap, but they are not interchangeable.

That distinction causes most confusion. A GPU may work with the Linux amdgpu kernel module and drive a desktop perfectly while remaining unavailable to the ROCm compute runtime used by Ollama. Conversely, rocminfo may recognize a device that Ollama's packaged ROCm version does not officially target.

Check the hardware before changing anything:

Look for a gfx identifier and the marketing name. If rocminfo is missing, crashes, or reports no GPU agent, Ollama is downstream of the real problem. Fix the driver, device permissions, or ROCm installation first.

VRAM imposes a separate limit. A supported GPU cannot hold a model whose weights and working buffers exceed available memory, so Ollama may place only some layers on the GPU and leave the rest in system RAM. Quantization reduces the weight size, but context length also consumes memory through the key-value cache. This is why one model can run entirely on a Radeon card while a larger model on the same configured machine becomes a mixed GPU and CPU workload.

It installs more than Ollama. On supported AMD systems, the current Linux installation path supplies Ollama and the ROCm-side libraries it expects, but it does not replace every kernel driver or repair an unsupported Ubuntu configuration.

The official Ollama Linux download page↗ is the right starting point because packaging and filenames have changed across releases. Avoid old tutorials that instruct you to combine a latest Ollama binary with an arbitrary ROCm archive from another version. That mixture can leave the application running while its GPU backend fails to load.

A normal Linux installation uses the vendor script:

Read a remote script before running it on a managed machine. The script detects the platform, installs the relevant files, creates or configures the service, and may check for NVIDIA or AMD hardware. It cannot make an unsupported Radeon target supported.

The kernel-side amdgpu module must already expose the card correctly. The Ollama service account also needs access to the relevant render devices, commonly /dev/dri/renderD128 and /dev/kfd; group membership and distribution rules determine that access. A shell user successfully running rocminfo does not prove that the systemd service has identical permissions or environment variables.

After installation, inspect both layers:

For manual ROCm installation or a distribution-specific driver stack, follow AMD's ROCm installation documentation↗ rather than copying commands written for a different Ubuntu release. Tuxedo OS and other Ubuntu derivatives may share package ancestry without matching the supported kernel, repository, or driver combination exactly.

Ask Ollama directly. While a model is loaded, ollama ps reports its processor placement and can distinguish an all-GPU model from a split GPU and CPU allocation.

Run a prompt in one terminal, then execute this in another while the model remains available:

The PROCESSOR column is the useful evidence. 100% GPU indicates that the loaded model is assigned to the GPU backend, a GPU/CPU percentage indicates partial offload, and 100% CPU means the model is not being accelerated by the AMD device. Idle utilization in radeontop is less conclusive because short prompts can finish between display refreshes.

Logs explain the decision. On a systemd installation, use:

For a foreground diagnostic session, stop the service first and enable debug logging for the server process:

Then run the model from another terminal. Look for detected devices, available VRAM, selected backend, library-loading errors, and layer allocation. Starting a second server without stopping the service usually produces an address-in-use error instead of a useful test.

Do not treat HSA_OVERRIDE_GFX_VERSION as a compatibility switch. The variable can make an unsupported device identify as another GFX target, but it cannot add missing instructions, repair a mismatched driver, or create kernels compiled for the actual architecture. It is an experiment for specific near-compatible chips, not a sound default.

The decision rule is simple: if rocminfo cannot see a GPU agent, fix ROCm; if it can but Ollama cannot, inspect Ollama's packaged backend and service environment; if ollama ps shows a split, compare model size, context length, and available VRAM. Windows has its own backend packaging and service behavior, so a Linux ROCm fix should not be transplanted blindly. For that case, use the Ollama GPU troubleshooting guide for Windows.