Reference

LLM VRAM Requirements

How much memory each model needs to load — checked against the memory your hardware will actually hand it. Those are different numbers, and the gap is why people buy the wrong machine.

Last reviewed 2026-09-22 · Next review 2026-12 · Reviewed quarterly

Short answer: a model's weights need roughly 2 GB per billion parameters at FP16, 1 GB at 8-bit, and 0.5 GB at 4-bit, plus overhead. But advertised memory is not usable memory: a discrete GPU loses about 1GB to display and driver context, and a unified-memory machine only offers roughly 75% of its pool to a model because the OS lives in the same memory. A 128GB Strix Halo box tops out near 96GB for the GPU — a platform limit, not a setting you can change.

Memory required, and the smallest box that holds it

Model4-bit (Q4)8-bit (Q8)FP16Smallest box that fits (4-bit)
Llama 4 Scout 8B8B5 GB9 GB16 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
Llama 4 Maverick 70B70B40 GB75 GB140 GBApple Mac Mini M5 Pro64GB unified (max M5 Pro config)48GB usableCheck price
Llama 4 Behemoth 405B405B230 GB430 GB810 GBApple Mac Studio M5 Ultra512GB unified (M5 Ultra max config)384GB usableCheck price
DeepSeek R1 7B7B4.5 GB8 GB14 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
DeepSeek R1 70B70B40 GB75 GB140 GBApple Mac Mini M5 Pro64GB unified (max M5 Pro config)48GB usableCheck price
Gemma 3 9B9B6 GB10 GB18 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
Gemma 3 27B27B16 GB29 GB54 GBApple Mac Mini M4 Pro24GB unified18GB usableCheck price
Qwen 3 7B7B4.5 GB8 GB14 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
Qwen 3 72B72B42 GB77 GB144 GBApple Mac Mini M5 Pro64GB unified (max M5 Pro config)48GB usableCheck price
Stable Diffusion XL3.5B4 GB6 GB8 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
Flux.1 Dev12B8 GB13 GB24 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
Whisper Large V31.5B1.5 GB2.5 GB4 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
Mistral 7B7B4.5 GB8 GB14 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
Phi-4 14B14B9 GB15 GB28 GBIntel Arc B580 12GB12GB GDDR611GB usableCheck price
CodeLlama 34B34B20 GB36 GB68 GBNVIDIA GeForce RTX 409024GB GDDR6X23GB usableCheck price

Figures are weights plus loading overhead. "Smallest box that fits" is the lowest-usable-memory item in our catalog that clears the 4-bit figure — not the fastest, and not the cheapest. Check the configuration line — several of these machines ship in memory tiers, and only the configuration named clears the requirement. A buy link lands on the product's listing, which is often the base tier, so you may need to configure the memory up. Datacenter cards (A100, H100) are listed in the table below but excluded from this column: they fit, but they are not a recommendation a normal buyer can act on. Speed is a separate question: see the tokens/sec reference.

Advertised memory vs. memory a model can actually use

The column that matters when you are deciding what to buy is the third one.

HardwareAdvertisedUsableWhy
Apple Mac Studio M5 Ultra512GB unified (M5 Ultra max config)Check price512 GB384 GBmacOS reserves a working set from the shared pool. At 512GB even the 25% reservation leaves more usable memory than any other single desktop — this is the only entry here that holds frontier-scale MoE weights. Note Apple ships the 512GB configuration from late October 2026; 256GB is the launch maximum.
GMKtec EVO-X2 (Ryzen AI Max+ 395)128GB LPDDR5X (Ryzen AI Max+ 395)Check price128 GB96 GBStrix Halo exposes at most ~96GB of its 128GB pool to the GPU — a platform limit, not an OS reservation. This is the gap that catches buyers comparing it to a 128GB spec sheet.
NVIDIA DGX Spark128GB LPDDR5X coherent (GB10)Check price128 GB96 GBGrace Blackwell coherent memory — CPU and GPU address one pool, so the usable share is high, but the OS still resides in it.
Apple Mac Studio M5 Max128GB unified (max M5 Max config)Check price128 GB96 GBSame 128GB ceiling as the M4 Max it replaces, so the same models fit. The M5 Max gain is prompt processing, not capacity.
Apple Mac Studio M4 Max128GB unified (max M4 Max config)Check price128 GB96 GBmacOS reserves a working set from the shared pool; GPU-wired memory is capped below total by default. 128GB is the maximum M4 Max configuration (36GB standard, configurable to 48/64/128GB).
NVIDIA A100 80GB PCIe80GB HBM2eCheck price80 GB79 GBDatacenter card, no display output — reserve is driver context only.
NVIDIA H100 PCIe 80GB80GB HBM3Check price80 GB79 GBDatacenter card, no display output — reserve is driver context only.
Apple Mac Mini M5 Pro64GB unified (max M5 Pro config)Check price64 GB48 GBmacOS reserves a working set from the shared pool. At 64GB this is the cheapest Apple machine that clears a 70B model at Q4 with context headroom.
NVIDIA GeForce RTX 509032GB GDDR7Check price32 GB31 GBDedicated VRAM; ~1GB reserved for display + driver context.
NVIDIA GeForce RTX 409024GB GDDR6XCheck price24 GB23 GBDedicated VRAM; ~1GB reserved for display + driver context.
NVIDIA GeForce RTX 309024GB GDDR6XCheck price24 GB23 GBDedicated VRAM; ~1GB reserved for display + driver context.
Apple Mac Mini M4 Pro24GB unifiedCheck price24 GB18 GBmacOS reserves a working set from the shared pool; GPU-wired memory is capped below total by default.
NVIDIA GeForce RTX 508016GB GDDR7Check price16 GB15 GBDedicated VRAM; ~1GB reserved for display + driver context.
NVIDIA GeForce RTX 5070 Ti16GB GDDR7Check price16 GB15 GBDedicated VRAM; ~1GB reserved for display + driver context.
NVIDIA GeForce RTX 4080 SUPER16GB GDDR6XCheck price16 GB15 GBDedicated VRAM; ~1GB reserved for display + driver context.
NVIDIA GeForce RTX 5060 Ti 16GB16GB GDDR7Check price16 GB15 GBDedicated VRAM; ~1GB reserved for display + driver context.
NVIDIA GeForce RTX 4060 Ti 16GB16GB GDDR6Check price16 GB15 GBDedicated VRAM; ~1GB reserved for display + driver context.
Intel Arc B580 12GB12GB GDDR6Check price12 GB11 GBDedicated VRAM; ~1GB reserved for display + driver context.

What these numbers exclude

KV cache grows with context. The figures above cover weights plus loading overhead at a short prompt. Every token you keep in context adds key/value cache on top, and at long context that can add many gigabytes. The exact amount depends on the model's layer count, KV-head count and head dimension — architecture details that vary per model. We do not publish a per-model KV table here because we have not verified those configs against primary sources, and a wrong number is worse than no number. Plan headroom above the figure in the table, especially at 32k context and up.

Fitting is not the same as being fast. A model that fits in memory can still run slowly if memory bandwidth is low. Capacity decides whether it runs; bandwidth decides how fast. The tokens/sec reference covers the second question with measured, source-linked figures.

Multi-GPU changes the ceiling, not the math. Two 24GB cards hold a larger model than one, but layer-split inference adds interconnect overhead. Rows marked "needs multi-GPU" are beyond any single item in our catalog.

How the usable figure is calculated

Two rules, applied uniformly, so you can check any row yourself:

  • Discrete GPU: usable = VRAM − 1GB (display output and driver/CUDA context).
  • Unified memory: usable = total × 0.75 — the OS occupies the same pool. On macOS the GPU-wired limit is raisable, so treat this as a conservative planning figure rather than a hard ceiling.
  • Documented hard caps win. Where a platform imposes a lower ceiling than the rule above, that ceiling is used instead — Strix Halo's ~96GB GPU allocation out of a 128GB pool is the case that matters today.

We publish no measured claim on this page. Every figure is either a hardware spec or the output of the arithmetic above. Measured throughput lives on the benchmark reference, where each number cites its source.

Know the model but not the budget? Use the GPU Advisor to match a model and budget to hardware, or browse the full hardware catalog.

Disclosure: product links on this page are affiliate links. We may earn a commission if you make a purchase — at no extra cost to you.