How much VRAM do you need to run a 30B model locally?
Short answer: Running a 27B–34B model needs 16–20 GB of VRAM at 4-bit, 29–36 GB at 8-bit and 54–68 GB at FP16 — weights plus loading overhead, before any context. The smallest machine in our catalog that clears the 4-bit figure is the NVIDIA GeForce RTX 4090 — 24GB GDDR6X — which hands a model about 23 GB.
Last reviewed 2026-09-22 · Next review 2026-12 · The 30B band is where a single 24GB card starts to run out at anything above 4-bit. Covers the 27B and 34B models we hold requirement data for; there is no 32B entry in our catalog, so treat 27B as the floor and 34B as the ceiling of this band.
Memory required, by model and quantization
| Model | Parameters | 4-bit (Q4) | 8-bit (Q8) | FP16 |
|---|---|---|---|---|
| Gemma 3 27B | 27B | 16 GB | 29 GB | 54 GB |
| CodeLlama 34B | 34B | 20 GB | 36 GB | 68 GB |
Weights plus loading overhead at a short prompt. 4-bit (Q4_K_M) is the quantization most people actually run; 8-bit is near-lossless; FP16 is what you need for training or for serving unquantized. Full table for all 15 models we hold data for: VRAM requirements reference.
Which GPUs and boxes actually fit it at 4-bit
Every buyable machine in our catalog whose usable memory clears 20 GB — the largest 4-bit figure above, so everything listed holds every model on this page, including the biggest. Usable, not advertised: a discrete GPU loses about 1 GB to display and driver context, and a unified-memory machine offers roughly 75% of its pool because the OS lives in the same memory.
| Hardware | Advertised | Usable | Headroom over 20 GB |
|---|---|---|---|
| NVIDIA GeForce RTX 409024GB GDDR6XCheck price | 24 GB | 23 GB | +3 GB |
| NVIDIA GeForce RTX 309024GB GDDR6XCheck price | 24 GB | 23 GB | +3 GB |
| NVIDIA GeForce RTX 509032GB GDDR7Check price | 32 GB | 31 GB | +11 GB |
| Apple Mac Mini M5 Pro64GB unified (max M5 Pro config)Check price | 64 GB | 48 GB | +28 GB |
| GMKtec EVO-X2 (Ryzen AI Max+ 395)128GB LPDDR5X (Ryzen AI Max+ 395)Check price | 128 GB | 96 GB | +76 GB |
| NVIDIA DGX Spark128GB LPDDR5X coherent (GB10)Check price | 128 GB | 96 GB | +76 GB |
Showing the 6 tightest fits; 3 larger machines in the catalog also clear it. Ordered tightest fit first, not best buy — headroom is what you have left for context, and more of it is what lets you run long prompts without spilling to system memory. Datacenter cards (A100, H100) are excluded: they fit, and they are not an answer a normal buyer can act on. Fitting is not the same as being fast — speed is bandwidth, and that lives on the tokens/sec reference.
How these numbers are derived
Weights take roughly 2 GB per billion parameters at FP16, 1 GB at 8-bit and 0.5 GB at 4-bit, plus about 20% for loading overhead. Usable memory is VRAM − 1 GB on a discrete GPU and total × 0.75 on unified memory, with documented platform caps winning where one exists — Strix Halo exposes about 96 GB of its 128 GB pool to the GPU.
KV cache is on top of this. The figures cover weights at a short prompt. Every token you hold in context adds key/value cache, and at 32k context and up that is gigabytes. We do not publish a per-model KV table because the architecture details it needs are not verified against primary sources here, and a wrong number is worse than no number. Treat the headroom column as your context budget.
No measured claim appears on this page. Every figure is a catalog spec, a requirement from our model memory table, or the output of the arithmetic above.
Other sizes
Further reading
- Best Consumer GPU for Local LLM 2026 — Buyer's Guide (RTX 5090 / 4090 / 3090, B580, Apple Silicon)
- Cheapest 32GB GPU for Local LLMs 2026: Intel Arc Pro B70 vs RTX 5090 vs Dual RTX 3090
- Best GPU for a Local Coding Assistant in 2026: VRAM Tiers to Replace Copilot with Qwen3-Coder, GLM-5.2 & Kimi K2.7 Code
Know the model but not the budget? Use the GPU Advisor to match a model and a budget to hardware, or browse the full hardware catalog.
Disclosure: product links on this page are affiliate links. We may earn a commission if you make a purchase — at no extra cost to you.