How much VRAM do you need to run a 70B model locally?
Short answer: Running a 70B-class model needs 40–42 GB of VRAM at 4-bit, 75–77 GB at 8-bit and 140–144 GB at FP16 — weights plus loading overhead, before any context. The smallest machine in our catalog that clears the 4-bit figure is the Apple Mac Mini M5 Pro — 64GB unified (max M5 Pro config) — which hands a model about 48 GB.
Last reviewed 2026-09-22 · Next review 2026-12 · Covers the 70B–72B dense models we hold requirement data for. Mixture-of-experts models with a 70B total parameter count behave differently — only the active experts are hot, but all weights still have to be resident.
Memory required, by model and quantization
| Model | Parameters | 4-bit (Q4) | 8-bit (Q8) | FP16 |
|---|---|---|---|---|
| Llama 4 Maverick 70B | 70B | 40 GB | 75 GB | 140 GB |
| DeepSeek R1 70B | 70B | 40 GB | 75 GB | 140 GB |
| Qwen 3 72B | 72B | 42 GB | 77 GB | 144 GB |
Weights plus loading overhead at a short prompt. 4-bit (Q4_K_M) is the quantization most people actually run; 8-bit is near-lossless; FP16 is what you need for training or for serving unquantized. Full table for all 15 models we hold data for: VRAM requirements reference.
Which GPUs and boxes actually fit it at 4-bit
Every buyable machine in our catalog whose usable memory clears 42 GB — the largest 4-bit figure above, so everything listed holds every model on this page, including the biggest. Usable, not advertised: a discrete GPU loses about 1 GB to display and driver context, and a unified-memory machine offers roughly 75% of its pool because the OS lives in the same memory.
| Hardware | Advertised | Usable | Headroom over 42 GB |
|---|---|---|---|
| Apple Mac Mini M5 Pro64GB unified (max M5 Pro config)Check price | 64 GB | 48 GB | +6 GB |
| GMKtec EVO-X2 (Ryzen AI Max+ 395)128GB LPDDR5X (Ryzen AI Max+ 395)Check price | 128 GB | 96 GB | +54 GB |
| NVIDIA DGX Spark128GB LPDDR5X coherent (GB10)Check price | 128 GB | 96 GB | +54 GB |
| Apple Mac Studio M5 Max128GB unified (max M5 Max config)Check price | 128 GB | 96 GB | +54 GB |
| Apple Mac Studio M4 Max128GB unified (max M4 Max config)Check price | 128 GB | 96 GB | +54 GB |
| Apple Mac Studio M5 Ultra512GB unified (M5 Ultra max config)Check price | 512 GB | 384 GB | +342 GB |
Ordered tightest fit first, not best buy — headroom is what you have left for context, and more of it is what lets you run long prompts without spilling to system memory. Datacenter cards (A100, H100) are excluded: they fit, and they are not an answer a normal buyer can act on. Fitting is not the same as being fast — speed is bandwidth, and that lives on the tokens/sec reference.
How these numbers are derived
Weights take roughly 2 GB per billion parameters at FP16, 1 GB at 8-bit and 0.5 GB at 4-bit, plus about 20% for loading overhead. Usable memory is VRAM − 1 GB on a discrete GPU and total × 0.75 on unified memory, with documented platform caps winning where one exists — Strix Halo exposes about 96 GB of its 128 GB pool to the GPU.
KV cache is on top of this. The figures cover weights at a short prompt. Every token you hold in context adds key/value cache, and at 32k context and up that is gigabytes. We do not publish a per-model KV table because the architecture details it needs are not verified against primary sources here, and a wrong number is worse than no number. Treat the headroom column as your context budget.
No measured claim appears on this page. Every figure is a catalog spec, a requirement from our model memory table, or the output of the arithmetic above.
Other sizes
Further reading
Know the model but not the budget? Use the GPU Advisor to match a model and a budget to hardware, or browse the full hardware catalog.
Disclosure: product links on this page are affiliate links. We may earn a commission if you make a purchase — at no extra cost to you.