Ollama silently truncates your context — how to tell, and how to fix it

Your model's card says 128k or 256k context. Your actual sessions get about 4k. Nothing warns you — the transcript is clipped from the top before the model ever sees it, so the model "forgets" earlier instructions, loses names it was told, or contradicts its own earlier answers. People blame the model. It's the config.

Why it happens

The context window is set per-request. When a client (or the CLI) doesn't ask for one, Ollama falls back to a small default of a few thousand tokens — regardless of what the model natively supports. The model's advertised maximum is what it can do, not what it's being given.

How to verify in one minute

Paste ~8k tokens of text containing a distinctive fact near the top, then ask about that fact. If the model has no idea, your effective window is smaller than your prompt. You can also watch the server log — a line like truncating input prompt is the smoking gun.

The fix: set num_ctx — but not to the maximum

The KV cache is reserved from num_ctx up front, and it lives in VRAM. Request the model's full 256k and you'll blow past your VRAM and spill to CPU, trading truncation for a 5× slowdown. The right value is the largest context that still fits 100% on your GPU — a number that depends on your card, your model, and your quant, so it has to be measured, not guessed. Then bake it in so every client gets it:

# Modelfile
FROM qwen3.5:9b
PARAMETER num_ctx 32768

ollama create qwen3.5-tuned -f Modelfile

Skip the manual sweep: ollama-tune measures GPU placement across a ladder of context sizes on your machine, finds the largest all-GPU value, and flags truncation risk. Free report; the licensed tier bakes the tuned model for you.