How much VRAM do you need to run a 7B or 8B model locally?
Short answer: Running a 7B or 8B model needs 4.5–5 GB of VRAM at 4-bit, 8–9 GB at 8-bit and 14–16 GB at FP16 — weights plus loading overhead, before any context. The smallest machine in our catalog that clears the 4-bit figure is the Intel Arc B580 12GB — 12GB GDDR6 — which hands a model about 11 GB.
Last reviewed 2026-09-22 · Next review 2026-12 · The entry scale for local inference — every model here loads on a single mid-range consumer GPU at 4-bit. Figures are for the dense 7B/8B releases we hold data for.
Memory required, by model and quantization
| Model | Parameters | 4-bit (Q4) | 8-bit (Q8) | FP16 |
|---|---|---|---|---|
| Qwen 3 7B | 7B | 4.5 GB | 8 GB | 14 GB |
| DeepSeek R1 7B | 7B | 4.5 GB | 8 GB | 14 GB |
| Mistral 7B | 7B | 4.5 GB | 8 GB | 14 GB |
| Llama 4 Scout 8B | 8B | 5 GB | 9 GB | 16 GB |
Weights plus loading overhead at a short prompt. 4-bit (Q4_K_M) is the quantization most people actually run; 8-bit is near-lossless; FP16 is what you need for training or for serving unquantized. Full table for all 15 models we hold data for: VRAM requirements reference.
Which GPUs and boxes actually fit it at 4-bit
Every buyable machine in our catalog whose usable memory clears 5 GB — the largest 4-bit figure above, so everything listed holds every model on this page, including the biggest. Usable, not advertised: a discrete GPU loses about 1 GB to display and driver context, and a unified-memory machine offers roughly 75% of its pool because the OS lives in the same memory.
| Hardware | Advertised | Usable | Headroom over 5 GB |
|---|---|---|---|
| Intel Arc B580 12GB12GB GDDR6Check price | 12 GB | 11 GB | +6 GB |
| NVIDIA GeForce RTX 508016GB GDDR7Check price | 16 GB | 15 GB | +10 GB |
| NVIDIA GeForce RTX 5070 Ti16GB GDDR7Check price | 16 GB | 15 GB | +10 GB |
| NVIDIA GeForce RTX 4080 SUPER16GB GDDR6XCheck price | 16 GB | 15 GB | +10 GB |
| NVIDIA GeForce RTX 5060 Ti 16GB16GB GDDR7Check price | 16 GB | 15 GB | +10 GB |
| NVIDIA GeForce RTX 4060 Ti 16GB16GB GDDR6Check price | 16 GB | 15 GB | +10 GB |
Showing the 6 tightest fits; 10 larger machines in the catalog also clear it. Ordered tightest fit first, not best buy — headroom is what you have left for context, and more of it is what lets you run long prompts without spilling to system memory. Datacenter cards (A100, H100) are excluded: they fit, and they are not an answer a normal buyer can act on. Fitting is not the same as being fast — speed is bandwidth, and that lives on the tokens/sec reference.
How these numbers are derived
Weights take roughly 2 GB per billion parameters at FP16, 1 GB at 8-bit and 0.5 GB at 4-bit, plus about 20% for loading overhead. Usable memory is VRAM − 1 GB on a discrete GPU and total × 0.75 on unified memory, with documented platform caps winning where one exists — Strix Halo exposes about 96 GB of its 128 GB pool to the GPU.
KV cache is on top of this. The figures cover weights at a short prompt. Every token you hold in context adds key/value cache, and at 32k context and up that is gigabytes. We do not publish a per-model KV table because the architecture details it needs are not verified against primary sources here, and a wrong number is worse than no number. Treat the headroom column as your context budget.
No measured claim appears on this page. Every figure is a catalog spec, a requirement from our model memory table, or the output of the arithmetic above.
Other sizes
Further reading
Know the model but not the budget? Use the GPU Advisor to match a model and a budget to hardware, or browse the full hardware catalog.
Disclosure: product links on this page are affiliate links. We may earn a commission if you make a purchase — at no extra cost to you.