Real-world LLM inference benchmarks
Theoretical performance is easy to compute — actual tokens/sec depends on your backend, quant kernel and driver. This database pairs the computed ceiling for every model, GPU and quantization with measured numbers people actually recorded. Filter it, sort it, and add your own run.
| Model | Params | VRAM | Fits | Tok/s (theoretical) | Tok/s (measured) | Power | $/1M tok |
|---|
Theoretical tok/s is a memory-bandwidth roofline (how it's computed) — a ceiling, not a promise; real throughput is lower. $/1M tokens uses indicative on-demand cloud rates. Measured values are community-contributed and cross-checked before they're marked verified. Not sure which quantization to run? See FP16 vs Q8 vs Q6 vs Q4.
Contribute a benchmark
Ran a model on your own hardware? Add it. We'd rather publish ten honest rows than a thousand guesses, so every submission is cross-checked before it goes in.
Copy the JSON below and send it via contact. (The maintainer can also point this form at a collector to accept submissions directly.)
Evidence & method
How this calculation works
Presents derived efficiency figures for GPUs, such as the memory-bandwidth decode ceiling, computed from published specifications rather than measured runs.
Data sources
- NVIDIA, AMD & Intel GPU documentation
- Bitpute Methodology
Assumptions
- Ranked from published specifications: VRAM, memory bandwidth, TDP and price class
- Efficiency proxies are derived (e.g. bandwidth-bound decode ceiling = bandwidth ÷ weight size)
- Single GPU, inference-oriented unless stated
Limitations
- Derived proxies are not a substitute for measured benchmarks on your workload
- Real performance depends on framework, model, batch, context and drivers
- Rankings shift as prices and availability change
- For comparison and orientation, not a performance guarantee
Data status: Hardware specs — vendor documentation. Rankings — computed, not measured.
Related
Why does this estimate differ from other calculators?
- Computed proxies vs measured runs
- Which spec dimensions are weighted
- Price and availability snapshots
- Rounding