"Ollama is slow" almost always resolves to one of four measurable causes. Here's how to tell which one you have — and why the fix people try first (tweaking kernels/backends) is usually the one that can't work.
The KV cache is reserved up front from your requested context size. Past some context, model + cache no longer fit in VRAM and llama.cpp silently places layers on the CPU. Generation drops from ~50 tok/s to single digits. Check while a chat is loaded:
curl -s http://localhost:11434/api/ps
If size_vram < size, you're spilling. Fix: find the largest context that
stays 100% on GPU and cap there (details).
Each generated token reads essentially the whole quantized model from VRAM. A 6.6 GB model on a ~384 GB/s card tops out near 384/6.6 ≈ 58 tok/s — physics, not configuration. If your measured speed is already near that ceiling, kernel tweaks and backend swaps won't move it. The real levers are a smaller quant, a smaller model, or speculative decoding.
If a client doesn't request a context size, Ollama uses a small default (~4k). Long conversations get clipped before the model sees them — users describe it as the model getting slow or forgetful mid-session. It's neither: it's truncation (how to verify).
A game, a training run, another model instance. Symptoms: speeds vary wildly between runs. Measure on an idle card before concluding anything.
Every cause above shows up in about three minutes of measurement: sweep context sizes,
read placement from /api/ps at each step, record tok/s, compare against your
card's bandwidth ceiling.