How Bitpute calculates VRAM, speed & cost
Every number on this site is computed from a published formula — not scraped, averaged, or guessed. This page shows the exact math, the constants, and what we deliberately don't model.
Why Bitpute exists
Choosing a GPU for an AI model shouldn’t come down to a single VRAM number. The memory a model needs moves with its parameter count, the precision it is stored at, the quantization you pick, runtime overhead and — where the architecture is published — the KV cache that grows with context length. A model that looks like it fits inside a card’s advertised VRAM can still leave too little usable memory to run in practice.
Bitpute exists to make those trade-offs visible. Rather than issuing one generic recommendation, it computes the underlying requirement stage by stage and compares it against usable GPU capacity — so you can see why a model fits, doesn’t fit, or is a tight match, and change one variable to find out what would.
Who publishes this and how it is reviewed: About Bitpute and the editorial policy.
* KV cache is included when the required architecture data (layers, KV heads, head dimension) is available for the model. Where it is not, the figure shown is the base footprint — weights plus overhead — and the page says so.
All calculators here run client-side. Nothing about your hardware or the models you look up is sent to a server. The trade-off is that our numbers are estimates from first principles, not measured benchmarks — accurate enough to plan a purchase, but always sanity-check against a real run for your exact stack.
1Model weights
The dominant cost is the weights themselves. Given a model's parameter count and the precision it's stored at:
params(B) is billions of parameters. bpw is bits per weight — set by the quantization. We divide by 8 to go from bits to bytes, and by 2³⁰ (1,073,741,824) because we report in binary gigabytes (GiB), the unit GPUs actually allocate in. On the rest of the site we write the shorter GB, the way GPU vendors and everyone else label a card — but the arithmetic is always binary, so a figure shown as 4.5 GB is the 4.52 GiB computed here.
2Runtime overhead
Loading a model costs more than the raw weights: the CUDA context, the memory allocator, and working buffers all take space. We model this as a fixed base plus a small fraction of the weights:
For the 8B example above that's 0.75 + 0.05 × 4.52 ≈ 0.98 GiB, so weights + overhead ≈ 5.5 GiB before any context.
3KV cache (context)
Every token you feed or generate is cached as key/value tensors. This grows linearly with context length and sits on top of the weights — at long context it can rival or exceed them:
The leading 2 is for the two tensors (K and V). bytes defaults to 2 (an FP16 cache). Models using grouped-query attention (GQA) have far fewer kv_heads than attention heads, which is why a modern 8B model's cache is small while older architectures balloon.
4Total footprint & single-GPU fit
A card doesn't hand you 100% of its VRAM — the display, drivers and fragmentation take a slice. So we treat usable VRAM = capacity × 0.95, and a model fits when total ≤ usable. For a comfortable daily driver we suggest capacity × 1.10 headroom over the total, so you're not living at the edge of OOM.
5Decode speed ceiling
Token generation is memory-bandwidth bound: each token requires reading the active weights from VRAM once. So the theoretical upper bound on tokens/second is:
Real throughput is lower — kernel efficiency, batching, sampling, and CPU offload all cost tokens. Treat it as "this card physically cannot exceed X for this model," useful for comparing cards, not for promising a number.
6Power & running cost
Wall-plug draw is higher than the GPU's rated TDP once you add the CPU, board and PSU losses, so we estimate system power ≈ 1.3 × GPU TDP. Energy cost is then power × hours × your electricity rate (the calculator defaults are editable). Buy-vs-rent compares that running cost plus hardware amortization against on-demand cloud pricing.
7Quantization reference
Bits per weight for the six formats we model:
| Format | Bits / weight | Typical use |
|---|---|---|
| FP16 | 16.0 | Full precision — max quality, max VRAM |
| Q8_0 | 8.5 | Near-lossless |
| Q6_K | 6.56 | Very high quality |
| Q5_K_M | 5.69 | High quality, balanced |
| Q4_K_M | 4.85 | The common sweet spot |
| Q4_0 | 4.5 | Smallest — some quality loss |
These are representative effective sizes. K-quants are mixed — different tensors in the same file get different types — so the exact bits-per-weight varies with architecture. For a concrete reference point, llama.cpp publishes measured figures for Llama 3.1 8B in its quantize documentation: Q4_K_M 4.8944, Q5_K_M 5.7036, Q6_K 6.5633, Q8_0 8.5008, F16 16.0005. We round to the values above and treat them as typical rather than exact; on a 70B model the difference between 4.85 and 4.8944 bpw is roughly 0.4 GB, which is inside the headroom the fit calculation already reserves.
8Data sources & assumptions
- Model specs (parameters, layers, KV heads, head dim, native context) come from each model's official card and
config.json. - GPU specs (VRAM, memory bandwidth, TDP) come from manufacturer datasheets.
- Prices are indicative — India street prices for hardware, on-demand rates for cloud — and move constantly. We show them to compare orders of magnitude, not to quote you a checkout total. Currency conversion uses a fixed reference rate stated on the cost tools.
9What we deliberately don't model
Being explicit about the edges matters more than pretending there are none:
- Activation memory at large batch sizes (we assume batch 1 for local single-user inference).
- Framework-specific allocator behavior and fragmentation beyond the flat overhead term.
- Tensor-/pipeline-parallel communication overhead across multiple GPUs (the multi-GPU planner approximates this separately).
- Full fine-tuning optimizer states — we model LoRA/QLoRA footprint as ≈ 1.5 × inference, not full AdamW training.
- Speculative decoding, FlashAttention variants, and other runtime tricks that shift real speed.
10How these numbers are verified
There is no single named reviewer signing off on prose here — instead numbers are held to a mechanical parity check on every build, and every input constant is traceable to a primary source:
- Parity test. An automated test asserts that the numbers baked into all 500+ pages match the reference engine (
vw-engine.js) to within 0.01 GB, across models from 0.1B to 671B. It runs after any change to a formula, constant or the GPU catalogue, and fails the build on drift. - Manufacturer-datasheet grounding. GPU specifications — VRAM capacity, memory bandwidth, TDP — come directly from the vendor datasheets cited in the references section, not from third-party rankings or crowd-sourced figures. Model specifications come from each model’s published card and
config.json. Where a figure cannot be verified from a primary source, the page says so rather than estimating. - One source of truth. The browser calculators and the pre-computed pages run the identical formulas, so a page can never quietly drift from the interactive tool.
This is deliberate: for a calculator, a reproducible method you can re-run beats a credential you have to take on faith. How content is produced and corrected is set out in the editorial policy.
Common mistakes when sizing VRAM
Every one of these has a number behind it elsewhere on this site. They are listed because each is a way a correct-looking calculation produces a wrong answer.
- More VRAM does not always mean faster. The RTX 4060 Ti 16GB holds bigger models than the RTX 3060 12GB but has lower memory bandwidth (288 GB/s against 360 GB/s). Decode speed tracks bandwidth once a model fits, so the larger card is roughly 20% slower on the models both can run.
- A 24 GB card does not give you 24 GB. We size against 95% of nameplate VRAM — about 22.8 GB on a 24 GB board. The display buffer, CUDA context and allocator fragmentation take the rest. Sizing to the nameplate figure is the most common way a model that “should fit” throws an out-of-memory error.
- Two cards with the same name can have different specs. An A100 80GB SXM runs at 2,039 GB/s and 400 W; the PCIe card is 1,935 GB/s at 300 W. H100 is starker: 3.35 TB/s SXM against roughly 2 TB/s PCIe. Most cloud instances are PCIe. Our GPU pages state which module the figures describe.
- Mixture-of-experts models still need all the weights resident. Active-parameter counts describe how much compute runs per token, not how much memory is occupied. A 671B model with 37B active still needs the full 671B in VRAM. Sizing an MoE by its active count understates the requirement by an order of magnitude.
- Instruct and base checkpoints have identical footprints. Fine-tuning changes weight values, not weight counts. If a base model fits, its instruct sibling fits. Where our catalogue holds both, each page names the other.
- Context length can cost more than the model. KV cache grows linearly with context. A 3B model at Q4_K_M is under 2 GB of weights, but a long context window can add more than that again. Model size alone does not tell you whether a setup fits.
- Quantization bits-per-weight are not exact integers. Q4_K_M is a mixture — different tensors get different types — so effective size lands near 4.85–4.89 bits per weight depending on architecture, not 4.0. Sizing a 4-bit build at 4 bits underestimates it by roughly 20%.
11References & sources
The formulas above rest on public, primary sources:
- Transformer attention & the KV cache — Vaswani et al., Attention Is All You Need (2017); per-token key/value memory follows directly from the multi-head attention it defines.
- Quantization bit-widths — the GGUF k-quant formats (Q4_K_M, Q5_K_M, Q6_K, Q8_0…) as implemented in llama.cpp
quantizereference; our bits-per-weight figures are the effective sizes of those formats. - Model specifications — each model's official card and
config.jsonon Hugging Face model cards (parameters, layers, attention heads, head dimension, native context). - GPU specifications — manufacturer datasheets for VRAM, memory bandwidth and TDP: the NVIDIA A100, H100, H200 and L40S datasheets, and the GeForce/RTX product pages. Where a chip ships in more than one form factor we use the SXM module figures and say so on the GPU page, because SXM and PCIe differ in both bandwidth and TDP (A100 80GB: 2,039 GB/s @400 W SXM vs 1,935 GB/s @300 W PCIe; H100: 3.35 TB/s @700 W SXM vs ~2 TB/s @350 W PCIe).
Every constant is chosen from these; if you have a better primary source for one, tell us.
12Changelog
The full version history for the engine, methodology and data now lives on the dedicated changelog page. In brief: the current calculation engine is v1.1.0 (July 2026), parity-tested to ±0.01 GB.
Found an error or have a better source for a constant? Corrections are welcome — we'd rather fix a number than defend it. See the engineering notes for deeper derivations.