Model guide
What GPU do you need for Llama 3.3 70B?
In short
- Llama 3.3 70B Instruct needs about 42.6 GB at Q4_K_M (weights plus runtime overhead; KV cache is extra).
- Smallest card that fits: RTX A6000 (48 GB).
- 5 of 12 cards in our set hold it at this quantization.
You need about 45.1 GB at Q4_K_M with 8K context. That means a 48 GB card (RTX A6000, L40S) at the absolute edge, an 80 GB card comfortably — or two 24 GB consumer cards with the model split across them, which llama.cpp and Ollama both support.
Architecture at a glance
Parameters
70.6B
Layers
80
KV heads
8
Head dim
128
Max context
128K
KV @8K
2.5 GB
The 70B is where single consumer cards stop. A 4090's 24 GB covers barely half the requirement, so no quantization on this page rescues it — even Q2-class quants leave you tight with no context room. The realistic local path is 2×3090/4090; the realistic single-card path is 48 GB or more. At the full 128K context the KV cache alone is 40 GB, pushing the total past 82 GB — a single H100 no longer fits it, and you're into H200 or multi-GPU.
VRAM by quantization (8K context)
| Quant | Total VRAM | Cheapest GPU that fits |
|---|---|---|
| Q4_K_M | 45.1 GB | RTX A6000 |
| Q5_K_M | 52.4 GB | A100 80GB |
| Q6_K | 59.9 GB | A100 80GB |
| Q8_0 | 76.6 GB | H200 141GB |
| FP16 | 141.3 GB | Multi-GPU / H200+ |
Weights and KV cache are exact arithmetic from the model's published config; overhead (0.75 GB + 5% of weights) is a calibrated estimate. "Fits" means at most 95% of the card. Method on the Engineering Center.
Compatible GPUs at Q4_K_M
Green fits comfortably, amber is tight, faded doesn't fit — each links to that card's full page.
Can an RTX 4090 run Llama 3.3 70B?
No. Llama 3.3 70B needs about 45.1 GB at Q4_K_M, well past the 4090's 24 GB. Split it across two cards, or step up to bigger hardware.
What about maximum context?
At the full 128K window, the KV cache grows to about 40.0 GB and the total to 82.6 GB. The KV cache is the part that grows — the weights never change. To see the exact split at any context, run this model through the GPU memory calculator, check an Ollama tag in the Ollama calculator, or size a fine-tune in the training memory calculator.
Running it in the cloud
For rented hardware the sensible floor is the L40S — the smallest datacenter card that holds this model comfortably at Q4_K_M. No consumer card holds it alone, so cloud rental means datacenter instances or a multi-GPU node. Hourly prices move weekly, so we don't print them here — the workspace carries the current figures and weighs rental against electricity for this exact model.
Related models
8.03B params · 6.5 GB at Q4_K_M
Same family — the natural size step.
See requirements →72.7B params · 46.3 GB at Q4_K_M
Closest size in another family.
See requirements →32.8B params · 22.2 GB at Q4_K_M
Closest size in another family.
See requirements →Exact memory figures for every quantization: How much VRAM does Llama 3.3 70B Instruct need?
Common questions
What is the minimum GPU for Llama 3.3 70B?
At the default Q4_K_M quantization with an 8K context, Llama 3.3 70B needs about 45.1 GB of VRAM, so the practical minimum is a RTX A6000. Weights and KV cache are exact arithmetic from the model's config; a small runtime overhead estimate is included.
How much VRAM does Llama 3.3 70B need at maximum context?
At the full 128K window, the KV cache grows to about 40.0 GB and the total to 82.6 GB.
Can an RTX 4090 run Llama 3.3 70B?
No. Llama 3.3 70B needs about 45.1 GB at Q4_K_M, well past the 4090's 24 GB. Split it across two cards, or step up to bigger hardware.
Before you buy
Compare the shortlisted cards head to head in GPU Compare, size the exact context you need in the GPU Memory Calculator, and read which quantization to run before committing to a card — dropping one format down often removes the need for the next tier up.