Multi-GPU planner
Plan the rig
before you build it.
2 × 4090 or 4 × H100 — see what a multi-card rig actually delivers: pooled VRAM, real decode speed after tensor-parallel overhead, wall power, and the full hardware cost with a buy-vs-rent call.
Pooled VRAM
Weights shard across cards — near-linear, minus overhead.
Honest speed
Decode scales sub-linearly; we apply TP efficiency.
Full cost
N× hardware + power, vs the same rig on cloud.
—
—
How we model a rig
No linear-scaling myths.
VRAM
Weights and KV shard across cards, so capacity is N × per-card minus per-card runtime overhead. Near-linear, not perfect.
Speed
N × bandwidth ÷ weights × TP-eff. Tensor-parallel decode loses to all-reduce comms — ~0.9× at 2 cards, ~0.8× at 4, ~0.68× at 8.
Power & cost
Wall draw N × TDP × 1.3; hardware N × price; cloud N × $/hr. Break-even from the same cited cost model.
One big card or many small ones?
Size the whole cluster.
A rig only makes sense if the model needs it. The Workspace ranks single cards first — come here when one won't hold the job.
Evidence & method
How this calculation works
Estimates a model's total footprint, then computes how many GPUs of a given card are needed when it will not fit on one, using usable VRAM per card.
Data sources
- NVIDIA, AMD & Intel GPU documentation
- Hugging Face model cards
- Official model papers
- Bitpute Methodology
Assumptions
- Weights = parameters × bits-per-weight ÷ 8
- Runtime overhead = 0.75 GB + 5% of weights (CUDA context, allocator, buffers)
- Recommended VRAM = total × 1.10 (10% headroom)
- Usable VRAM = card capacity × 0.95
- KV cache counted only when the model architecture is known; when applied, batch size 1, FP16 KV, context as entered
- Splits the model across identical GPUs, sized by usable VRAM per card
Limitations
- Actual VRAM varies by framework (PyTorch, llama.cpp, vLLM, TGI), driver/CUDA version and OS
- Quantization implementations differ; real bits-per-weight can vary from the nominal value
- When architecture is unknown the KV cache is omitted, so long-context use will exceed the estimate
- Estimates are for planning and comparison, not a guarantee
Data status: Hardware specs — vendor documentation. Model metadata — community model cards, not independently verified.
Related
Why does this estimate differ from other calculators?
- Different footprint formulas and rounding
- Different KV-cache assumptions (batch, context, precision)
- Framework and inference-engine differences
- Reserved and fragmented VRAM
- Driver and CUDA overhead
- Precision and quantization choices