LLM Training Memory Calculator
How much GPU memory it takes to train a model, not just run it. Compare full fine-tuning, LoRA and QLoRA — with the optimizer and gradient state that inference calculators leave out — and see which GPUs can actually do the job.
Total memory
6.06 GB
Trainable params
6.8M
0.08% of base
Base weights
4.21 GB
Memory breakdown
Base weights and the gradient/optimizer state are exact for the mixed-precision Adam recipe. Activations are an estimate — the term most sensitive to your framework and sequence length.
Worth knowing
- QLoRA keeps the base model in 4-bit and trains small adapters, which is what lets large models fine-tune on a single GPU.
Can you train it?
- FitsRTX 3060 12GBconsumer5.54 GB free
- FitsRTX 4060 Ti 16GBconsumer9.54 GB free
- FitsRTX 4070 Ti SUPERconsumer9.54 GB free
- FitsRTX 3090consumer17.5 GB free
- FitsRTX 4090consumer17.5 GB free
- FitsRTX 5090consumer25.5 GB free
- FitsA100 40GBdatacenter33.8 GB free
- FitsRTX A6000workstation41.5 GB free
- FitsL40Sdatacenter41.8 GB free
- FitsA100 80GBdatacenter73.8 GB free
- FitsH100 80GBdatacenter73.8 GB free
- FitsH200 141GBdatacenter135 GB free
Methodology
How the numbers are computed
Full fine-tuning. Mixed-precision AdamW stores, for every trainable parameter, an fp16 weight, an fp16 gradient and three fp32 tensors — the master weight, Adam momentum and Adam variance. That is sixteen bytes per parameter, the well-known figure from the ZeRO paper and Hugging Face's training-anatomy documentation.
LoRA. The base model is frozen — its weights are held at your chosen precision but carry no gradient or optimizer state. Only the low-rank adapters are trained, at the full sixteen bytes each, and their count is derived from the rank and the dimensions of the targeted linear layers. Because adapters are a small fraction of the parameters, the frozen base dominates memory.
QLoRA. Same as LoRA, but the frozen base is quantized to 4-bit NF4 (about 4.5 effective bits per weight with double quantization). This is what shrinks a 70B base to roughly 40 GB and lets it fine-tune on a single 48 GB GPU.
Activations. Activation memory accumulates across all layers and scales with batch size and sequence length. Gradient checkpointing recomputes most of it instead of storing it. This term genuinely depends on the framework and attention kernel, so it is an engineering estimate, labeled as such — the same treatment inference overhead gets on this site.
Units. 1 GB here means 1 GiB (1024³ bytes), matching nvidia-smi. Multi-GPU figures assume weights, gradients and optimizer state shard across GPUs (ZeRO/FSDP-style) while activations stay per-GPU.
FAQ
About this calculator
- Why does full fine-tuning need so much more memory than inference?
- Inference holds only the weights and the KV cache. Training additionally stores a gradient for every trainable parameter and the optimizer's state. With mixed-precision AdamW that is sixteen bytes per parameter — an fp16 weight, an fp16 gradient, and three fp32 tensors (master copy, momentum, variance). That is why full fine-tuning a 7B model needs well over 100 GB, several times its inference footprint.
- How does QLoRA fit a 70B model on a single GPU?
- QLoRA freezes the base model and stores it in 4-bit (NF4, about 4.5 bits per weight), then trains small low-rank adapters that are typically well under 1% of the parameters. Only those adapters carry gradients and optimizer state, so the memory is dominated by the compact 4-bit base. A 70B base in 4-bit is roughly 40 GB, which is why it fits on a single 48 GB card — the result the QLoRA paper demonstrated for a 65B model.
- What does gradient checkpointing change?
- During the backward pass, training normally keeps the intermediate activations of every layer. Gradient (activation) checkpointing stores only a few and recomputes the rest, trading a little extra compute for a large memory saving. In this calculator turning it on cuts the activation term several-fold; it is standard practice for fine-tuning and is on by default.
- How accurate is the trainable-parameter count for LoRA?
- It is computed from the model's architecture — rank times the dimensions of each targeted linear layer, across all layers. Attention-only (q, v) is exact from the config; the all-linear option involves the MLP intermediate size, which is approximated as 3.5× hidden when it is not in the model record, so that figure is indicative. Either way the base weights dominate total memory, so the headline number stays reliable.
- Which term is an estimate and which is exact?
- Base weights and the gradient plus optimizer state are exact arithmetic for the mixed-precision Adam recipe. Activation memory is the estimated term — it genuinely varies with the framework, attention implementation and sequence length — so it is labeled as an estimate throughout, the same way this site treats runtime overhead for inference.
Planning inference rather than training? The GPU Memory Calculator covers weights and the KV cache for serving, and the Infrastructure Workspace ties memory, speed and cost together.
Why these numbers dwarf the inference figures: where training memory goes.