Ollama VRAM Calculator
Will your Ollama model fit your GPU?
Pick the model you'd pull, the quantization tag, your card and how long a context you keep — and see straight away whether it fits, with the weights, KV cache and overhead broken out. The numbers come from each model's real architecture, not a rule of thumb.
Configuration
Total VRAM
—
Weights
—
KV cache
—
Overhead (est.)
—
Weights and KV cache are exact arithmetic from each model's config; overhead (0.75 GB + 5% of weights) is a calibrated estimate. See the methodology.
Want cost and speed too? Open the full workspace, or the focused GPU memory calculator. Planning to train rather than run? Use the training memory calculator. Prefer a written answer for one model? The model GPU guides cover each one.
How much VRAM do I need to run a model in Ollama?
The answer comes down to three numbers: how big the model is, which quantization tag Ollama pulls, and how long a context you keep open. The calculator above works all three out for you, but here is the shape of it. At Ollama's default Q4_K_M tag, a 7B or 8B model lands around 6 to 7 GB and runs happily on an 8 GB card. A 14B sits near 10 GB. A 32B needs roughly 20 GB, so a 24 GB card handles it with room for context. A 70B at Q4_K_M wants about 42 to 45 GB, which puts it on a 48 GB workstation card or a data-center GPU.
Those figures are weights plus a KV cache plus a small runtime overhead. The weights are fixed once you pick a quant; the KV cache is the part that grows as you raise the context.
Which quantization does Ollama use by default?
When you type ollama run llama3.1 with no tag, Ollama pulls Q4_K_M. That format stores about 4.85 bits per weight, not a flat 4, because the block scales that keep the quantization accurate also take space. It is the sweet spot most people want: close to full quality, roughly a quarter of the FP16 size. If you have the memory to spare, q8_0 is nearly lossless at about double the footprint; if you are squeezed, the smaller K-quants trade a little quality for a smaller file. Switch the quant selector above to see each one measured against your card.
Sizing the rest of the machine? See how much system RAM you need.
Does Ollama offload to system RAM if the model does not fit?
It does. When a model is larger than your VRAM, Ollama keeps as many layers on the GPU as will fit and runs the rest on the CPU using system RAM. Nothing crashes, but the CPU layers are the bottleneck, and generation slows to a crawl once a meaningful share of the model is offloaded. The verdict above is deliberately strict about this: it tells you the full GPU-only requirement, so you can pick a card that holds the whole model and keeps every token fast.
Why does a longer context cost more memory?
Every token in the context window is stored as a key and a value in each layer of the model — that store is the KV cache. Double the context and you double that cache. On a small model it is a rounding error; on a 70B at 128K tokens it can add several gigabytes on top of the weights. The num_ctx you set in Ollama drives this directly, so if a model fits at 8K but not at 128K, the context is why. Change the context selector above and watch the cyan KV segment of the bar grow.
Common questions
Why does a longer context need more VRAM in Ollama?
The KV cache grows with context length. Every token you keep in the window is stored as key and value tensors across every layer, so raising the context from 8K to 128K can add several gigabytes on a large model. The calculator recomputes the KV cache each time you change the context so you can see the cost before you set num_ctx.