Local LLM VRAM Calculator: Match a Model to Your GPU
Estimate local LLM VRAM from parameters, bit width and context, then choose a quantized model that fits your laptop GPU.
Key points
- Local LLM memory is weights plus KV cache, runtime overhead and headroom, not just download size.
- A 7B model at 4-bit needs about 3.26 GiB for weights before context memory.
- KV cache grows with layers, KV heads, head dimension, context tokens and cache precision.
- Shared-memory laptops split one pool between the operating system, display and model.
- The calculator predicts memory fit, not tokens per second or benchmark performance.
How Much VRAM Does a Local LLM Need?
A local LLM's VRAM requirement is the weight storage, KV cache, runtime buffers and a working margin added together. The download size alone is not a fit test. A model can look comfortable in its quantized file and still run out of memory after you raise the context window.
Quantization stores each model weight with fewer bits. A 4-bit model uses four bits per weight instead of the 16 bits common in FP16 model files. KV cache is different: it is the attention state the model keeps for earlier tokens so it can process a new token. Context is the token window the model may use at once.
We use two equations in the LLM VRAM Calculator. Weight memory is parameters × bits per weight ÷ 8. KV cache memory is 2 × layers × KV heads × head dimension × context tokens × bytes per value, where the leading 2 covers keys and values. Add runtime buffers and free-memory headroom after that.
As of October 2026, our rolling analytics show 21 sessions and 0 calculations run on the LLM VRAM Calculator. That gap told us the form needed a guide beside it. The arithmetic below is the same logic we use to turn a GPU capacity into a memory-fit verdict.
Turn Three Inputs Into a Fit Decision
The arithmetic is easier to audit than a model-runner badge. You need the parameter count, weight precision and maximum context, then one more check against the memory your desktop leaves free.
Step 1: Calculate the weight footprint
Multiply the parameter count by the bits per weight, divide by 8, then convert bytes to GiB by dividing by 1,073,741,824. A 7B model at 4 bits therefore needs about 3.26 GiB before any cache is allocated. Using 8 bits doubles that weight figure, which is why a card that fits one tier may miss the next. Quantization scales and file metadata add a small amount, so treat the weight result as a lower-bound estimate.
Step 2: Size the KV cache
A model with 32 layers, 8 KV heads, a head dimension of 128 and FP16 cache uses 128 KiB for every token. At 4K tokens that is 0.50 GiB; at 32K it is 4.00 GiB; at 256K it is 32.00 GiB. If the model uses grouped-query attention, count its KV heads rather than its full attention-head count.
Step 3: Add overhead and test the real load
Weights and cache are the two large terms, but the runtime needs working buffers too. Use this order so a failure is traceable:
- Write down the GPU's advertised memory and the amount free after the desktop loads.
- Calculate weights at the quantized bit width you plan to download.
- Add KV cache for the largest context you will actually use, not the smallest.
- Reserve a margin for the display, browser, model runner and compute buffers.
- Load one model, set that context limit, run a long prompt and watch peak memory.
What 0.5B to 8B Models Need
The table separates weights from context memory so you can combine any row with any compatible tier. Values are in GiB and rounded. The KV columns assume FP16 cache with 32 layers, 8 KV heads and a head dimension of 128; substitute your model's real configuration before treating a total as a verdict.
| Parameters | 4-bit weights | 8-bit weights | KV at 4K | KV at 32K | KV at 256K |
|---|---|---|---|---|---|
| 0.5B | 0.23 GiB | 0.47 GiB | 0.50 GiB | 4.00 GiB | 32.00 GiB |
| 1B | 0.47 GiB | 0.93 GiB | 0.50 GiB | 4.00 GiB | 32.00 GiB |
| 1.5B | 0.70 GiB | 1.40 GiB | 0.50 GiB | 4.00 GiB | 32.00 GiB |
| 3B | 1.40 GiB | 2.79 GiB | 0.50 GiB | 4.00 GiB | 32.00 GiB |
| 7B | 3.26 GiB | 6.52 GiB | 0.50 GiB | 4.00 GiB | 32.00 GiB |
| 8B | 3.73 GiB | 7.45 GiB | 0.50 GiB | 4.00 GiB | 32.00 GiB |
For an 8 GiB card, a 7B model at 4 bits with 4K context starts at 3.76 GiB for weights and cache, leaving room for a light desktop. The same 7B model at 8 bits with 32K context reaches 10.52 GiB and needs at least a 12 GiB card, plus headroom. A 24 GiB card can hold an 8B model at 4 bits with 32K context; the same model at 256K context still needs 35.73 GiB, so it fails.
The useful pattern is not “bigger models always need more quantized VRAM.” Context can dominate. That is why the calculator asks for all three inputs and why a 0.5B model at 256K context can need more cache memory than a 7B model at 4K.
Work the table backward when the machine is fixed. If a 16 GiB card has 14 GiB free, subtract 4 GiB for the example 32K cache and the weight budget is about 10 GiB. That fits an 8B model at 8 bits, which needs 7.45 GiB, but not the same model at 16-bit precision. Repeat the subtraction whenever the context setting changes, because the cache term moves with it.
Where the Estimate Goes Wrong
The common mistake is comparing a quantized model file with total GPU memory and treating a narrow fit as success. The error often appears only after a long prompt, when the runner allocates the configured context or processes more concurrent tokens. Watch peak memory during a realistic session, not just while the model loads.
Integrated GPUs share system memory, while discrete GPUs have their own memory, as MDN's WebGPU overview explains. On a shared-memory laptop, the operating system, display and other applications draw from the same pool. Confirm the actual allocation policy in the laptop specification instead of assuming that every advertised gigabyte is available for the model.
Another edge case is GPU offloading. Shifting some layers to CPU RAM can make a model load on a smaller GPU, but transfers between the two memory pools add latency. The calculator's memory verdict is therefore a capacity check, not a prediction of tokens per second, prompt latency or concurrent-user capacity.
Cache precision is a fourth input that readers often hide inside the weight tier. If the runner stores keys and values at 8 bits instead of FP16, the cache can shrink, but the exact saving depends on that runner's implementation. Check the cache setting separately before you assume that a 4-bit model also uses a 4-bit cache.
Check these failure cases
- The runner reserves memory for the maximum context before the prompt fills it.
- Several simultaneous requests multiply the KV cache.
- Windows or macOS reserves shared graphics memory for the desktop.
- A draft model or extra vision encoder adds weights outside the main parameter count.
Choose the Quantization Tier That Fits
Start with the card's free memory, then work down from 8-bit to 4-bit only when the 8-bit total misses your margin. For a short-context chat workload on 8 GiB, a 7B or 8B model at 4 bits is the practical starting tier. With 12 to 16 GiB, 8-bit 7B becomes plausible and larger contexts become realistic. With 24 GiB, prioritize context and cache precision rather than assuming an 8B model will consume the whole card.
Model-runner fit labels and vendor calculators are convenient alternatives, but their assumptions can differ on context length, cache precision, offload and display reserve. Our calculator's limitation is the same one every sizing tool has: it cannot inspect the model runner's private buffers or benchmark its kernels. Treat the result as a first-pass capacity answer and verify it by loading the exact model and context you intend to use.
Once the memory question is settled, check the rest of the workload separately. Use the LLM Token Cost Calculator for API token pricing and the AI Water Cost Calculator for an order-of-magnitude energy and water estimate. Local inference avoids per-token API pricing, but it still depends on your hardware, electricity and usage time.
Write down the model name, quantization, context limit and peak memory after a successful test. That record is more useful than a single fit verdict when you change runners or update a model. It also separates a memory problem from a speed problem the next time a session disappoints.
Frequently asked questions
How much VRAM do I need for a local LLM?
Start with the model weights, then add KV cache and runtime headroom. A 7B model at 4-bit uses about 3.26 GiB for weights; 4K context adds roughly 0.50 GiB with the table's example architecture. An 8 GiB GPU can work with a light desktop, but an 8-bit model and larger context need more capacity.
What local LLM can I run with 8GB of VRAM?
A 7B or 8B model at 4-bit is a sensible starting point for short-context work on an 8 GiB card. With the example KV shape, 4K context adds 0.50 GiB to weights of 3.26 to 3.73 GiB. Leave room for the display, operating system and runtime, then test peak memory during a long prompt.
Is 16GB of VRAM enough for a local LLM?
Often yes, but the answer depends on precision and context. On a 16 GiB card, an 8B model at 8-bit uses about 7.45 GiB for weights, and 32K context adds 4.00 GiB with the table's assumptions. That leaves room for buffers, though larger contexts, concurrent requests or a heavy desktop can consume the margin.
Does 4-bit quantization cut local LLM VRAM use in half?
It roughly halves the weight memory compared with 8-bit, because each weight uses four bits instead of eight. It does not halve the full footprint. KV cache, runtime buffers and display reserve remain, and they can dominate at long context. Size the cache with its own precision rather than the weight width.
Can a VRAM calculator predict LLM speed?
No. A VRAM calculator estimates whether weights, KV cache and overhead fit in memory. Speed also depends on the GPU, memory bandwidth, compute kernels, batch size, context length and whether layers are offloaded. Use the memory result to avoid failed loads, then measure tokens per second with your exact runner and hardware.
References
- MDN Web Docs — WebGPU API · developer.mozilla.org