LLM VRAM Calculator: Who Benefits and Why
Reviewed by the OnlineFree.app team · Updated
Key points
- VRAM needed equals weight memory plus KV cache for your context plus runtime overhead.
- Weight memory scales linearly: Q4_K_M uses 4.8 bits per weight versus 16 for FP16.
- Doubling context length roughly doubles the KV-cache portion, not the whole estimate.
- A 24 GB card comfortably runs 30B-class models at Q4_K_M; 70B needs offload or two GPUs.
- Treat the output as a planning figure and confirm it in your runtime's memory report.
How much VRAM do you need to run a local LLM?
The short answer: weight memory, plus KV-cache memory for the context you will actually use, plus a runtime overhead of a few hundred megabytes to a couple of gigabytes. The LLM VRAM Calculator adds those three parts and prints a single figure in GB, along with a plain sentence saying which GPU tiers it clears.
Weight memory is the easy part because it scales linearly with parameter count and bits per weight. A 7B model stored at 4.8 bits per weight (Q4_K_M) needs about 4.2 GB — 7 billion × 4.8 ÷ 8. The same model in FP16 needs roughly 14 GB. Run 70B through the same arithmetic at Q4_K_M and you land near 42 GB, which is why the fit verdict matters more than the decimal place.
You give the tool three inputs — parameter count in billions, weight quantization, and context length — and it returns the total, a weights / context-KV / overhead breakdown, and a memory bar. We use the LLM VRAM Calculator as a first pass before downloading any model files, because a 20-second check beats a 40 GB download that cannot load.
Who benefits from the LLM VRAM Calculator
Three groups keep coming back to it: hobbyists choosing between an 8 GB and a 16 GB card, developers checking whether an existing card can host a second model alongside the one already running, and anyone pricing cloud GPUs who needs a sanity check before committing to an hourly rate.
For the buyer, the useful comparison is concrete. A 13B model at Q4_K_M is about 7.8 GB of weights, so with cache and overhead it lands near 9–10 GB — it will not load comfortably on 8 GB, but fits 12 GB with room. A 30B model at the same quantization is about 18 GB, which is a 24 GB card or nothing.
For teams, the same math applies to an A100 40 GB, an L40S 48 GB or an H100 80 GB; what changes is the price per hour, not the arithmetic. And because the whole thing runs in the browser on OnlineFree.app with no sign-up and no upload, you can test unannounced model sizes for a client without sending anything to a server.
Quantization is the biggest lever
Nothing moves the VRAM number faster than the quantization dropdown. FP16 uses 16 bits per weight, Q8_0 about 8.5, Q6_K 6.6, Q5_K_M 5.7, Q4_K_M 4.8, Q3_K_M 3.9 and Q2_K about 3. The fractional values are not a bug: k-quants keep some tensors at higher precision, so Q4_K_M really costs more than a flat 4 bits.
The savings compound quickly. A 13B model is roughly 26 GB at FP16, 13.8 GB at Q8_0 and 7.8 GB at Q4_K_M — going from FP16 to Q4_K_M cuts weight memory by about 3.3×, which is usually the difference between renting a 40 GB card and using the 24 GB one you own. A 70B model drops from about 140 GB to about 42 GB by the same step.
The honest caveat: Q4_K_M is the popular default for a reason, and Q2_K is visibly lossy on reasoning and long-form writing. Below 4 bits the memory you save per point of quality gets worse, and small models (1–3B) degrade faster than large ones at the same setting. Check a few prompts against a higher quantization before you commit.
Why context length quietly eats your headroom
The weights do not change when you raise the context, but the KV cache does. Every token in the window stores keys and values for every layer, so that portion of the estimate grows in a straight line with the number you pick.
The ratios are what matter. Moving from 4,096 to 32,768 tokens is eight times the cache; on a modern 7–8B model with grouped-query attention the KV portion at 4k is a few hundred megabytes, so at 32k it is a few gigabytes, and at 131,072 it can be the difference between fitting 24 GB and not. We have watched people blame the model when the context setting was the actual problem.
Set the context you will really configure, not the largest number the model advertises. llama.cpp-style backends reserve the KV buffer up front from the -c value, so a slider left at 131,072 costs memory even when your prompts are short. Quantizing the cache to 8-bit roughly halves that line, but the calculator does not model it — treat any gap as your safety margin.
Where the estimate is honest about its limits
The figure is a planning estimate, not a promise. Batch size above 1, activation buffers and backend-specific overhead all add memory the calculator does not itemise, and on Windows the desktop compositor can leave a 24 GB card reporting closer to 23 GB usable before your model starts.
Mixture-of-experts models are the other trap. Only a fraction of parameters are active per token, but all of them must be resident in VRAM, so size an MoE by its total parameter count rather than its active count. The tool's arithmetic is dense-model style; it is conservative for MoE in the right direction, but it will not match a back end's log line to the megabyte.
So verify. Load the model, then check nvidia-smi or the loader's reported buffer sizes, and keep 1–2 GB free on a desktop card. If you sketch deployment diagrams for a team, we usually pair this with Mermaid to SVG so the rig diagram and the numbers live in the same doc.
Check a card before you buy it
Read the verdict as tiers rather than a single yes/no. Under 8 GB covers 1–3B models and small-quant 7B; 12 GB handles 7–8B at Q4_K_M with a 4k–8k context; 16 GB adds 13B at Q4_K_M; 24 GB is the 30B tier at Q4_K_M, as of 2026 the most common prosumer ceiling.
Anything at 42 GB and above needs a 48 GB or 80 GB card, or two consumer cards with tensor parallelism — and that path carries its own overhead for communication buffers, so leave slack. Two used 24 GB cards are often cheaper than one 48 GB card, but only if your runtime splits the model cleanly.
For cloud rentals, the rule we use is simple: if the estimate sits above about 90% of an instance's memory, take the next tier up. Bursty batch sizes and longer prompts will not fit, and an out-of-memory crash mid-evaluation costs more than the extra hour.
Frequently asked questions
How accurate is the LLM VRAM Calculator's estimate?
It is a planning estimate, typically landing within a gigabyte or so of what a backend actually reserves, not a benchmark. It sums weights, KV cache for your stated context, and runtime overhead, but ignores batch size, backend quirks and KV quantization. Confirm the real figure in your loader's log or with nvidia-smi before relying on it.
Does the LLM VRAM Calculator send my inputs anywhere?
No. It runs entirely in your browser as a calculator, with no sign-up, no upload and no server round-trip, and it keeps working offline once loaded. That makes it safe for checking unreleased model sizes or client-specific hardware plans without sending anything over the network.
Can a 24 GB GPU run a 70B model at 4-bit?
No. A 70B model at Q4_K_M needs roughly 42 GB just for weights, before any cache or overhead. Even Q2_K at about 3 bits per weight lands near 26 GB, still over 24 GB. Running 70B means a 48 GB or 80 GB card, or two GPUs with tensor parallelism or CPU offload, which is much slower.
Why does context length change VRAM when the model size does not?
Because raising the context enlarges the KV cache, not the weights. Each token in the window stores keys and values for every layer, so the cache grows linearly with token count. Going from 4,096 to 32,768 tokens makes that portion eight times larger, which on smaller cards decides whether the model fits.
Does the estimate include the memory my operating system uses?
Only partly. The runtime overhead line covers typical backend and CUDA context costs, but not a desktop compositor or other GPU apps. On Windows a 24 GB card often shows about 23 GB free, so leave 1–2 GB of headroom and treat the calculator's total as a floor rather than a ceiling.