Model guide
What GPU do you need for Qwen2.5 32B?
In short
- Qwen 2.5 32B needs about 20.2 GB at Q4_K_M (weights plus runtime overhead; KV cache is extra).
- Smallest card that fits: RTX 3090 (24 GB).
- 9 of 12 cards in our set hold it at this quantization.
About 22.2 GB at Q4_K_M with 8K context — this is the largest mainstream model that still fits a 24 GB card, with about 1.8 GB of headroom on a 4090.
Architecture at a glance
Parameters
32.8B
Layers
64
KV heads
8
Head dim
128
Max context
128K
KV @8K
2.0 GB
That tight fit is the whole story of the 32B. It works at Q4_K_M and 8K; raise either and it doesn't. Q5_K_M needs 25.6 GB. The full 128K context adds 32 GB of KV cache for a 52 GB total, which is 48–80 GB card territory. If you own a 24 GB card, this model is the ceiling — run it at Q4, keep the context moderate.
VRAM by quantization (8K context)
| Quant | Total VRAM | Cheapest GPU that fits |
|---|---|---|
| Q4_K_M | 22.2 GB | RTX 3090 |
| Q5_K_M | 25.6 GB | RTX 5090 |
| Q6_K | 29.1 GB | RTX 5090 |
| Q8_0 | 36.8 GB | RTX A6000 |
| FP16 | 66.9 GB | A100 80GB |
Weights and KV cache are exact arithmetic from the model's published config; overhead (0.75 GB + 5% of weights) is a calibrated estimate. "Fits" means at most 95% of the card. Method on the Engineering Center.
Compatible GPUs at Q4_K_M
Green fits comfortably, amber is tight, faded doesn't fit — each links to that card's full page.
Can an RTX 4090 run Qwen2.5 32B?
Yes. At Q4_K_M with an 8K context, Qwen2.5 32B needs about 22.2 GB, leaving 1.8 GB of headroom on the 4090's 24 GB.
What about maximum context?
At the full 128K window, the KV cache grows to about 32.0 GB and the total to 52.2 GB. The KV cache is the part that grows — the weights never change. To see the exact split at any context, run this model through the GPU memory calculator, check an Ollama tag in the Ollama calculator, or size a fine-tune in the training memory calculator.
Running it in the cloud
For rented hardware the sensible floor is the A100 40GB — the smallest datacenter card that holds this model comfortably at Q4_K_M. Marketplace clouds also rent consumer cards; anything from the RTX 3090 up works for this model and usually costs less per hour. Hourly prices move weekly, so we don't print them here — the workspace carries the current figures and weighs rental against electricity for this exact model.
Related models
14.8B params · 11.0 GB at Q4_K_M
Same family — the natural size step.
See requirements →32.8B params · 22.2 GB at Q4_K_M
Same architecture, different training.
See requirements →27.2B params · 19.8 GB at Q4_K_M
Closest size in another family.
See requirements →Exact memory figures for every quantization: How much VRAM does Qwen 2.5 32B Instruct need?
Common questions
What is the minimum GPU for Qwen2.5 32B?
At the default Q4_K_M quantization with an 8K context, Qwen2.5 32B needs about 22.2 GB of VRAM, so the practical minimum is a RTX 3090. Weights and KV cache are exact arithmetic from the model's config; a small runtime overhead estimate is included.
How much VRAM does Qwen2.5 32B need at maximum context?
At the full 128K window, the KV cache grows to about 32.0 GB and the total to 52.2 GB.
Can an RTX 4090 run Qwen2.5 32B?
Yes. At Q4_K_M with an 8K context, Qwen2.5 32B needs about 22.2 GB, leaving 1.8 GB of headroom on the 4090's 24 GB.
Before you buy
Compare the shortlisted cards head to head in GPU Compare, size the exact context you need in the GPU Memory Calculator, and read which quantization to run before committing to a card — dropping one format down often removes the need for the next tier up.