Skip to content
Bitpute

The workspace

Size, price and benchmark a GPU setup on one screen

Change the model, quantization, GPU or context and everything recalculates at once — the memory fit, the inference speed, the cloud rental cost, the electricity bill, and a plain-English read on whether the setup makes sense. Every figure is sourced, and the ones that are estimates say so.

Configuration

KV cache precision
Cost assumptions
Fits comfortablyH100 80GB
45.1 GB / GPU

VRAM / GPU

45.1 GB

of 79.8 usable

Speed (single stream)

32.5

tokens/sec (est.)

Cloud rental

$1.87–$3.93

per hour on-demand

Local power

490 W

$43/mo electricity

Memory on the H100 80GB

H100 80GB · 80 GB45.1 / 80 GB
Weights 39.9 GBKV cache 2.50 GBOverhead 2.74 GB

Analysis

  • Llama 3.3 70B at GGUF Q4_K_M needs about 45.1 GB per GPU, and your H100 80GB has room to spare. This is a comfortable fit.
  • Expect roughly 21.9–43.0 tokens per second for single-stream generation, bounded by the H100 80GB's 3,350 GB/s memory bandwidth. Batching multiple requests raises total throughput well above this.
  • Renting this in the cloud runs about $1.87–$3.93 per hour on-demand ($1.4k–$2.9k a month at full utilisation). Running your own card instead costs about $43 a month in electricity at your rate — before the hardware itself.
View full report

Cards that fit this setup

Single-GPU, smallest sufficient first — the least wasteful choice for Llama 3.3 70B at GGUF Q4_K_M, 8,192 ctx.

  • A100 80GB
    $1.00/hr+
  • H100 80GBselected
    $1.87/hr+
  • H200 141GB
    $2.50/hr+
  • RTX A6000
    $0.45/hr+

How this was calculated

Weights
70.6B × 4.85 bpw ÷ 8
39.9 GB
KV cache
2 × 80 × 8 KV heads × 128 × 8,192 × 2B
2.50 GB
Overhead
0.75 GB/GPU + 5% of weights (estimate)
2.74 GB
Total footprint
45.1 GB

Confidence by metric

  • Memory (VRAM)

    Weights & KV cache are exact arithmetic

    Verified

  • GPU fit verdict

    Derived from official memory capacities

    Verified

  • Inference speed

    Roofline band from bandwidth, not benchmarked

    Estimated

  • Runtime overhead

    0.75 GB/GPU + 5% engineering estimate

    Estimated

  • Cloud cost

    Indicative market range, as of July 2026

    Estimated

Engine v0.6.1 · full methodology →

What each number means

Reading the workspace

VRAM per GPU is the memory each card must hold — weights, KV cache and runtime overhead. The bar shows it against the card's real capacity; the full arithmetic is in the "How this was calculated" panel.

Speed is a single-stream decode estimate from the GPU's memory bandwidth — the rate at which it can read the model's weights once per generated token. It is a realistic band, not a benchmark; running many requests at once raises total throughput well above it.

Cloud rental is an indicative on-demand range from public provider pricing, shown with the date it was gathered. GPU prices move week to week, so confirm the live rate before you commit spend. Local power is the electricity cost of running your own card at your rate — the hardware price is not included.