Ollama Not Using GPU Windows: Diagnose CPU Fallback

Check the loaded model first.

Run ollama ps while a response is being generated, because its PROCESSOR column distinguishes a model held entirely on the GPU from one split between GPU and CPU memory.

If PROCESSOR says 100% CPU, the diagnosis is settled. A split such as 48%/52% CPU/GPU means acceleration is active, but the model, context cache, or both did not fit in available VRAM. Do not judge this from Windows Task Manager's default 3D graph. In Task Manager, open Performance, select the GPU, and change one graph to CUDA or Compute if that engine is available.

For NVIDIA hardware, run this in PowerShell while Ollama is generating tokens:

Look for ollama.exe in the process list, rising memory use, and nonzero GPU utilization. A brief zero between prompts is normal. The NVIDIA utility reports management and utilization data through the documented System Management Interface↗.

The common symptom is simple: Ollama does not seem to be using the GPU because generation is slow, yet the installation produces valid answers. Prove CPU fallback before changing drivers or environment variables.

Support depends on the backend.

On native Windows, current Ollama releases can accelerate compatible NVIDIA GPUs through CUDA and selected AMD GPUs through ROCm, while unsupported graphics hardware falls back to CPU execution.

The official Ollama GPU compatibility documentation↗ is the authority for the current compute capability floor, AMD families, Vulkan status, and override variables. Read it against the exact adapter name shown under Task Manager > Performance > GPU or returned by nvidia-smi, not the laptop's marketing name.

Integrated graphics complicate matters. A Windows laptop may expose an Intel iGPU beside an NVIDIA RTX adapter, and Windows graphics settings can launch the Ollama application on the low-power device even though CUDA remains installed. Open Settings > System > Display > Graphics, add the Ollama executable if necessary, choose Options, select High performance, and restart Ollama from its tray icon.

Do not use CUDA_VISIBLE_DEVICES as a magic acceleration switch. That variable selects among GPU devices already visible to the runtime; it cannot make an unsupported adapter acquire CUDA support. An incorrect value can instead hide the only working NVIDIA GPU.

Match the precise GPU model.

A supported product family is insufficient when an older chip revision lacks the required CUDA compute capability, an AMD architecture is absent from the ROCm list, or the installed driver predates the runtime bundled with Ollama.

For NVIDIA, nvidia-smi must open without NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. If that exact error appears, repair or update the NVIDIA driver before touching Ollama. A Microsoft Basic Display Adapter entry in Device Manager is another driver failure, not an Ollama setting.

For AMD, compare the adapter against Ollama's Windows ROCm list. Similar names are not interchangeable. Radeon integrated GPUs and older consumer cards are frequent dead ends even when they provide DirectX acceleration for games or video playback.

Virtual machines, Remote Desktop sessions, and a Docker container add another boundary between Ollama and the device. If Ollama Docker container is not using the GPU, native Windows driver health proves only the host works; the container also needs an NVIDIA-compatible runtime and an exposed device. Diagnose native ollama.exe first. It removes an entire layer of failure.

You usually cannot force it.

The practical fix is to restore GPU discovery, restart the Ollama process, and load a model small enough for available video memory rather than setting an undocumented force RTX switch.

1. Quit Ollama from the notification-area icon.
2. Update the NVIDIA or AMD graphics driver from the hardware vendor, then restart Windows.
3. Confirm nvidia-smi works for NVIDIA, or confirm the exact AMD adapter is supported.
4. Launch Ollama again and run a small model.
5. Run ollama ps during generation and inspect PROCESSOR.

If Ollama will fail to discover your NVIDIA GPU after a driver update, check logs before reinstalling. Windows logs are under %LOCALAPPDATA%\Ollama; server.log is the useful file for runtime discovery messages. The vendor's Windows troubleshooting notes↗ also document log locations and debug startup behavior.

Quit the tray process before changing its environment. A variable set in a new PowerShell window does not retroactively alter an already running Ollama server. To collect verbose messages for one session, stop the existing process and run:

Then launch a model from a second terminal. Search the first terminal for CUDA, ROCm, library discovery, and available memory messages. Generic reinstall advice rarely fixes a GPU hidden by a stale process, broken driver, or unsupported architecture.

Video memory is probably exhausted.

A larger model, increased context length, parallel requests, or another application consuming VRAM can force some layers onto system memory even though GPU detection and the driver remain healthy.

Run ollama ps and compare SIZE with PROCESSOR. Then inspect dedicated GPU memory in Task Manager or nvidia-smi. Close games, Stable Diffusion interfaces, browsers using heavy WebGPU workloads, and other inference servers before loading the model again.

Context matters more than many users expect. Raising the context window allocates a larger key-value cache, so a model that previously fit can become a CPU/GPU split without any software update. Quantization changes the equation too: a smaller Q4 build may fit where a higher-precision variant does not.

Test with a smaller model before chasing the runtime. If that model reports 100% GPU, compatibility is fine and the original workload simply exceeds available VRAM. Reduce context length, unload competing models with ollama stop <model>, or choose a smaller quantization.

An old GPU with too little memory is not worth hours of registry edits. If every useful model spills heavily into CPU memory and performance remains unacceptable, replace the hardware or run the workload on a machine with more VRAM. For model-sizing context, see how to run an LLM locally.