Best CPU for Local LLM Inference in 2026: Why Memory Bandwidth Beats Core Count

Search "best CPU for local LLM" and you get chips ranked by Cinebench scores and gaming frame rates. Not one of those rankings mentions the single number that decides how fast tokens come out of your machine. The result is a predictable and expensive mistake: buyers spend an extra $300 on core count that does nothing, then discover the model they wanted does not fit anyway.

This post is the correction. One equation, applied consistently, decides every recommendation below — including the recommendation, in most cases, to not upgrade your CPU at all.

The Short Answer: Buy Bandwidth, Not Cores

Past roughly eight fast modern cores, core count stops changing your tokens per second on inference workloads. Memory bandwidth keeps changing it linearly, all the way up. So the buying question is never "which CPU is fastest" — it is "how many GB/s can this platform feed, and does my model fit in the fast memory".

"Adding CPU cores does not speed up token generation; adding memory bandwidth does." — Compute Market Team, editorial verdict, 13 September 2026

Why Token Generation Is a Memory-Bandwidth Problem

The mechanism is simple enough that you can do the arithmetic in your head, which is exactly why it should be in every CPU guide and is in almost none.

To produce one token, an autoregressive language model reads its active weights — every one of them, once. There is no way around this: the matrix multiplications that generate a token need the matrices, and the matrices live in memory. Then it does it again for the next token. Generation is therefore a loop that streams the model out of memory once per token, and the hardware limit is how fast memory can stream.

Work an example. A 7B dense model at 4-bit quantization is roughly 4GB of weights. Put it on a mainstream desktop with dual-channel DDR5-6000: two 64-bit channels at 6000 MT/s is 96 GB/s of theoretical peak bandwidth. 96 ÷ 4 = 24 tokens per second — and that is the ceiling, not the result. Real systems achieve somewhere around 70–80% of theoretical peak once you account for refresh, latency and controller overhead, so the honest number is closer to 17 tok/s, and measured community results on this class of hardware typically land lower still once KV cache traffic and sampling overhead are included.

Now scale the model instead of the hardware. The same machine running a 30B dense model at 4-bit is reading roughly 17GB per token: 96 ÷ 17 ≈ 5.6 tok/s theoretical, realistically around 4. Nothing about the CPU changed. The model got bigger, so the answer got slower, in direct proportion. That proportionality is the whole story, and it is why tokens per second is predictable from a spec sheet if you read the right line.

Two honest caveats. First, this describes generation. Prompt processing — the prefill pass that reads your input before the first token appears — is compute-bound, parallel, and does benefit from more cores and wider vector units. If you routinely feed 32K-token prompts, extra cores buy you a shorter wait for the first token and nothing after it. Second, techniques like speculative decoding change the effective bytes-per-token by generating several tokens per verification pass; they shift the constant in the equation, not the shape of it.

The Bandwidth Table: What Each Platform Actually Delivers

Here is the number that should be on every CPU comparison and is on none of them. Every figure is labelled vendor spec or theoretical peak, because the two get conflated constantly and the difference is 20–30%.

Read that table as a single sentence: a $2,000 desktop CPU platform delivers roughly 5% of an RTX 5090's memory bandwidth, and a six-year-old used RTX 3090 delivers nearly 10× what your DDR5 does. Bandwidth is what you are buying, and GDDR7 and GDDR6X exist precisely because system memory cannot do this job. The unified-memory platforms sit in the gap: two to five times a desktop, well short of a discrete card, and with vastly more capacity than either.

The Spillover Cliff: What Happens When Your Model Leaves VRAM

This is the section that decides most purchases, and the one the competing guides skip entirely.

Suppose a model runs comfortably on your GPU at 45 tok/s. You switch to a slightly larger quantization and now 10% of the layers do not fit in VRAM. Intuition says you lose about 10% of your speed. You do not. Because generation reads every active weight once per token, that 10% of layers is read over the PCIe bus from memory running at a fraction of the card's bandwidth — and the GPU waits for it every single token. The slow path sets the pace for the whole loop.

The mechanism produces a discontinuity rather than a slope. In practice, builders report dense-model configurations dropping from comfortable interactive speeds into low single-digit tokens per second the moment a meaningful slice of layers lands in system RAM — the exact figure depends on how much spills and on your memory bandwidth, but the shape is always the same: a cliff, not a ramp. You can predict the floor yourself with the equation, using system-RAM bandwidth and the bytes that live there.

That rule is why this post's most common recommendation is a GPU, not a processor. If your target is a 14–32B model, Phi-4 14B or Gemma 3 27B class, the money goes into a card. Our consumer GPU rankings for local LLMs and the AI GPU buying guide cover that decision properly, and the cheapest route to 32GB of VRAM is the post to read if the model is bigger than one 16GB card.

MoE Changed the Math (and Made the CPU Matter Again)

Everything above assumes a dense model, where "active weights" means "all the weights". Mixture-of-Experts models break that assumption, and that break is the reason the CPU answer is different in 2026 than it was in 2024.

An MoE model routes each token through a small subset of its expert tensors. A model with hundreds of billions of total parameters may activate only a few tens of billions per token. Return to the equation: total size sets your capacity requirement, but only the active fraction sets your bandwidth requirement. So parking rarely-touched expert tensors in cheap, slow system RAM while attention and shared tensors stay on the fast GPU stops being absurd and starts being the optimal layout.

Mainline llama.cpp exposes this directly with --n-cpu-moe N, which moves the MoE feed-forward tensors of the first N layers to the CPU while attention stays on the GPU. Tune N upward until the model fits in your VRAM, and no further. We covered the tuning, the starting values by card and the throughput numbers in depth in running 100B+ MoE models on a 16GB GPU — rather than repeat it, here is the part that changes what CPU and RAM you buy:

• It works on MoE, and only on MoE. A dense model has no rarely-used weights to park. Offloading one collapses throughput exactly as section four describes. Qwen 3 72B-class dense models are not candidates; the Qwen 3.5 and 3.6 MoE variants, DeepSeek V4-class models and the Llama 4 Scout and Maverick MoE variants are.
• System RAM capacity becomes a real spec, not a formality. You are now sizing RAM to hold expert tensors, which means 64GB minimum and 128GB if you are serious.
• System RAM bandwidth becomes a real spec too. The offloaded experts are still read per token. Dual-channel DDR5-6000 is the floor; this is the one desktop scenario where a quad-channel platform earns its price.
• Cores still do not matter. Even here, what you need is the memory controller. An 8-core chip with fast dual-channel memory beats a 16-core chip with slow memory.

Model-specific sizing lives in the individual hardware guides — Qwen 3.6, DeepSeek V4-Flash, GLM 5.2 and Kimi K2.6 all include the offload configuration that works for that model.

How Much System RAM — and What DDR5 Now Costs

Most RAM advice on the internet was written when 64GB cost $200. It does not any more, and the crossover calculations that advice implied have all moved.

As of 31 August 2026, a 64GB DDR5 kit ran $396–$607 against roughly $95 before the shortage, per Tom's Hardware's RAM Price Index; a 32GB DDR5-6000 kit was sitting around $375–$400 in early September 2026. The cause is supply-side rather than demand-side — DRAM manufacturers reallocated wafer capacity toward high-bandwidth memory for AI datacentre customers, shrinking the pool of conventional DDR5 reaching retail, as MSI's own memory-shortage explainer describes. We covered the consequences for build planning in the DRAM shortage buying guide.

That last row is the crossover worth staring at. At these prices, 128GB of DDR5 costs roughly what a used 24GB RTX 3090 costs at $699–$999 — and the 3090 delivers 936 GB/s against your DDR5's ~96 GB/s. If your model fits in 24GB, that comparison is not close. The RAM only wins when the model cannot fit in any card you can afford. Our RAM sizing guide for local AI works through the capacity side in detail, and build vs buy in 2026 covers how the memory spike shifted the whole build economy.

One genuine advantage of the high-bandwidth APU route is that it sidesteps the DDR5 spot market entirely. A GMKtec EVO-X2 at $1,999–$3,649 has its LPDDR5X — up to 128GB on the top SKU — priced into the machine, bought on a contract months ago, not off a shelf this month. In September 2026 that is a larger financial factor than most spec differences. The full review is in our Strix Halo mini PC deep dive and the Ryzen AI Max+ 395 review.

Recommended Builds by Budget

Four configurations, each derived from the equation rather than from a benchmark chart. Each one carries the honest line about who should not buy it.

1. The GPU-first tower — cheapest CPU that does not embarrass you

An 8-core Ryzen 7 or Core Ultra 5, a B-series board, 32GB of DDR5-6000, and every remaining dollar in the graphics card. This is the right build for most people reading this, and the CPU line is deliberately boring: RTX 5060 Ti 16GB at $429–$479 for models up to about 14B at 4-bit, or RTX 5070 Ti at $1,049–$1,299 if you want the 896 GB/s. Pair either with a Samsung 990 Pro — model weights are tens of gigabytes and load time is pure sequential read.

Who should not buy this: anyone whose target model is above 32B. You will spend the whole build and still hit the spillover cliff. Read the NVMe guide and the PSU guide before finalising the parts list.

2. The no-GPU bandwidth box — Ryzen AI Max+ 395

256 GB/s of quad-channel LPDDR5X and up to 128GB of it, in a box that draws a fraction of a tower's power. This is the only x86 consumer platform that turns "CPU inference" into a real answer rather than an apology, and the capacity means 70B-class models simply load. The GMKtec EVO-X2 is the mainstream way to buy it.

Who should not buy this: anyone who needs CUDA. The ROCm and Vulkan paths are workable for inference and thin for everything else, and if your target model fits in 16GB a $429 GPU is faster for a quarter of the price. See DGX Spark vs Strix Halo and Strix Halo vs Mac Studio M4 Max for the head-to-heads, and the mini PC for AI hub for the wider field.

3. The CUDA-native unified box — DGX Spark

128GB of coherent unified memory at 273 GB/s (NVIDIA vendor spec) with a 20-core Arm CPU and an intact CUDA stack, at $3,999. NVIDIA's IFA 2026 local-AI announcements on 3 September 2026 reinforced the direction of travel here, including a reported up-to-1.9× llama.cpp throughput gain on RTX 5090 and an RTX Spark N1X with a 128GB unified configuration slated for October — worth knowing before you commit, and covered in RTX Spark vs DGX Spark.

Who should not buy this: anyone optimising for tokens per dollar. This is a development and prototyping machine, and a Strix Halo box gets you the same memory capacity for roughly half the money if you can live without CUDA. Compare it honestly against a tower in DGX Spark vs RTX 5090.

4. The Apple route — highest bandwidth per watt

The Mac Studio M4 Max at $1,999–$5,999 quotes 546 GB/s of unified memory bandwidth with up to 192GB behind it — more bandwidth than an RTX 5060 Ti and six times the capacity of an RTX 5090. The Mac Mini M4 Pro at $1,399–$1,599 is the 273 GB/s, 24GB entry point. Both run MLX and llama.cpp well; MLX vs llama.cpp on Apple Silicon covers which runtime to use.

Who should not buy this: anyone doing training or fine-tuning, and anyone whose toolchain assumes CUDA. Start at the Apple Silicon for AI hub, then Mac Mini M4 Pro vs Mac Studio M4 Max to size the configuration, or Mac Studio M4 Max vs RTX 5090 for the capacity-versus-speed trade. On a budget, Mac Mini M4 Pro vs Beelink SER8 is the relevant entry-level comparison — the Beelink SER8 at $449–$599 runs DDR5-5600 in dual channel — about 90 GB/s of theoretical peak, so expect it to behave like the desktop rows at the top of the bandwidth table, not the unified-memory ones.

Budget mini PCs deserve one honest sentence rather than a section: the GMKtec M8 at $389–$459, GMKtec M6 Ultra at $429–$549 and Intel NUC 13 Pro at $600–$900 are all dual-channel machines — the DDR5 and LPDDR5 models land in the ~77–96 GB/s band, and the DDR4 NUC 13 Pro roughly half that. They run 7B models acceptably and anything larger slowly, by arithmetic, not by opinion. Our AI on a budget hub and the best mini PCs for AI rank that tier properly.

What Doesn't Matter (Stop Paying for It)

Short, blunt, and the section most likely to save you money.

• Core counts above 8–12. They cannot consume bandwidth that does not exist. A 9950X does not generate tokens faster than a 9700X on the same memory.
• High boost clocks. Same reason. You are waiting on memory, not on the ALU.
• PCIe 5.0 x16 vs x8 for a single inference GPU. Once weights are resident in VRAM, the bus carries activations and tokens, not the model. It matters for multi-GPU tensor parallelism and for loading time, not for single-card generation. See our PCIe explainer and the multi-GPU setup guide if you are actually going wide.
• AVX-512 as a purchase driver. Useful to the kernels, already handled by the runtime, and not the constraint on your throughput.
• Desktop NPUs and quoted TOPS figures. They do not carry large-model token generation in the mainstream local stacks today. A 50 TOPS badge is a laptop battery feature, not an inference spec. What an AI PC actually is covers why.
• Exotic RAM timings. Bandwidth is rate × channels. Latency tuning is a rounding error against that.

One thing that does matter and gets ignored: sustained power draw. A 575W TDP card under continuous inference is a different electricity bill from a 60–120W unified-memory box, and over three years that gap is real money — we costed it out here.

The Verdict

If your model fits in VRAM, buy the cheapest current 8-core CPU, 32GB of DDR5-6000, and put every remaining dollar into the graphics card. The CPU is not your bottleneck and no amount of it will become one in a useful direction.

If your model does not fit in VRAM, stop shopping for CPUs and start shopping for memory bandwidth attached to capacity — which in September 2026 means a Ryzen AI Max+ 395 box, a DGX Spark, or a Mac Studio, in that order of value. A desktop CPU upgrade is the one purchase that cannot solve this problem, and it is the one every generic roundup will sell you.

And if you are running MoE models specifically, the calculation genuinely is different: buy 64–128GB of the fastest system RAM your board supports, keep the GPU you have, and tune --n-cpu-moe. That is the single case in 2026 where the answer to "does the CPU matter" is yes.

Next steps: run your target model through the GPU advisor to see what it actually needs, check the benchmark database for measured numbers on the hardware you are considering, and start at the local LLM hub if you are still deciding what you want to run at all.