Learn
The concepts behind every number on this site, in plain English — no math degree required.
A practical comparison of FP16, Q8_0, Q6_K, Q5_K_M, Q4_K_M and Q4_0 — bits per weight, real VRAM for a 14B model, quality, and which to pick for your
System RAM vs VRAM for running LLMs: 16GB minimum, 32GB comfortable, 64GB+ for offloading big models. How offload budget works, the Apple unified-memo
The KV cache stores attention keys and values for every token in context, and grows linearly with context length — at long context it can exceed the m
Quantization stores model weights in fewer bits to cut VRAM without retraining. What it does, how much you save, why Q4 isn't exactly 4 bits, and when
For local LLM inference, VRAM capacity decides what you can run and VRAM bandwidth decides how fast. Why memory, not compute, is the bottleneck — and
Want the formulas instead? See the methodology or the engineering notes.
Start here
GPU concepts for running LLMs maps how these pieces relate — which specification decides what, and in what order.
Core definitions
The two variables every calculation on this site depends on: