How to Use the LLM VRAM Calculator, Step by Step
Reviewed by the OnlineFree.app team · Updated
Key points
- The LLM VRAM Calculator adds weights, KV cache and overhead into one GB figure with a GPU fit verdict.
- Weight memory is parameters × bits per weight ÷ 8, so 70B at Q4_K_M is 42 GB.
- KV cache scales linearly with context length, which is why 4K fits and 32K often does not.
- Q4_K_M really averages about 4.8 bits per weight, not a flat 4 — a gigabyte-scale difference at 70B.
- Leave 10–15% headroom over the reported figure and verify with your runtime's own memory report.
What does the LLM VRAM Calculator estimate?
The LLM VRAM Calculator estimates total GPU memory for a local language model as three added parts: the weight memory of the model file, the KV cache that grows with your context window, and a small runtime overhead. You supply parameter count, quantization and context length; it returns a single figure in GB plus a plain-language verdict on which common GPUs can hold it.
That breakdown matters because most people only think about the first part. A 7B model at Q4_K_M is 7 × 4.8 ÷ 8 = 4.2 GB of weights, and that number is what gets quoted in forum threads. The figure the calculator shows you is always larger, because the context window has to live in VRAM too.
We built it as a client-side page with no sign-up and no server round-trip. Type your numbers, read the answer, close the tab — nothing you enter leaves the browser, and the page keeps working offline once loaded. If you want to sanity-check a used GPU listing before you buy, that is the whole workflow.
Step by step: from parameter count to a fit verdict
Start with the model size field in billions of parameters, using the number printed on the model card: Llama 3.1 8B becomes 8, Mistral 7B becomes 7, Qwen 2.5 72B becomes 72. The quick-pick chips cover 0.5, 1.5, 3, 7, 8, 13, 30, 70 and 120, which covers most of what people actually download. Anything up to 2000 is accepted, and invalid input shows an inline error instead of a result.
Next pick the weight quantization. This is the biggest single lever: FP16 uses 16 bits per weight, Q8_0 about 8.5, Q5_K_M about 5.7, Q4_K_M about 4.8, Q3_K_M about 3.9 and Q2_K about 3. Multiply bits by parameters and divide by 8 to get weight GB — so 70B at Q4_K_M is 70 × 4.8 ÷ 8 = 42 GB, while the same model at Q8_0 is 74.4 GB. Notice that Q4_K_M is not exactly 4 bits; the 4.8 reflects the mixed precision of real GGUF files, and using a flat 4 would under-count by almost a gigabyte at 7B.
Finally set the context length — 512 up to 131072 tokens, with 4096 as the default — and read the result. You get the total in GB to one decimal, a compact line showing how it splits between weights, context-KV and overhead, a memory bar, and a sentence telling you which GPU tiers it clears. That verdict is the part worth reading twice.
Why does context length change VRAM so much?
The KV cache grows linearly with the number of tokens you keep in memory, so moving from 4,096 to 32,768 tokens multiplies that slice by eight, and 4,096 to 131,072 multiplies it by thirty-two. On models with grouped-query attention the per-token cost is modest; on older multi-head architectures it is dramatically higher. The LLM VRAM Calculator applies a representative per-token cost rather than reading your exact config.json, which is why the context component is an estimate and not a promise.
The practical consequence is that a model can look comfortable at 4K and collapse at 32K. A 13B Q4_K_M model has 7.8 GB of weights, which sounds fine on a 12 GB card until you push the window out and the KV cache eats the remaining room. Long-document summarisation, RAG pipelines and agent loops that accumulate history all sit in that trap.
Match the setting to your actual use: 2,048–4,096 for chat and short coding assistant work, 8,192–32,768 for file Q&A and long transcripts. Setting 131,072 "just in case" inflates the estimate and, if you try it for real, will usually OOM. Run the calculation twice with two different context values and compare — it teaches the trade-off faster than any explanation.
Reading the fit verdict on real GPU tiers
The verdict lines up your total against the consumer tiers people actually own: 8, 12, 16 and 24 GB, then the prosumer and datacentre cards above that. A 7B at Q4_K_M with a 4K window lands in the region of 5–6 GB, so an 8 GB card handles it with a little room to spare. An 8B at Q4_K_M is about 4.8 GB of weights alone, which puts an 8 GB card right at the edge — playable, but one browser tab with hardware acceleration can push it over.
12 GB is the comfortable tier for 7B–13B at Q4. 16 GB opens up 13B–14B with long context or a 30B at Q3_K_M. 24 GB (RTX 3090, 4090) is the sweet spot for 30B at Q4_K_M — 18 GB of weights leaves real headroom for context and overhead. Anything at 70B is out of reach: 42 GB of weights at Q4_K_M needs two 24 GB cards with tensor parallelism, or an A100 40GB-class card for the Q3 range.
Treat the verdict as a shopping filter, not a benchmark. It tells you whether a configuration is plausible before you spend an evening downloading weights. If you are also weighing cloud options, the same GB figure is what you use to pick an instance type — see the free tools on OnlineFree.app for the other quick sanity checks we keep alongside the LLM VRAM Calculator.
Common mistakes and where the estimate is rough
The most frequent error is entering the quantisation label's nominal bit count instead of its real one. If you assume Q4_K_M is 4.0 bits, a 70B model looks like 35 GB instead of 42 GB — a gap wider than an entire RTX 4090. Use the preset options rather than a hand-typed custom bits value unless you know the exact file.
The second mistake is forgetting that the GPU also drives your display. On Windows or a Linux desktop, the compositor and any hardware-accelerated browser can hold a gigabyte or more. On a 12 GB card that is the difference between loading and not loading. Batch size and backend workspaces (CUDA context, cuBLAS buffers) add more, and the calculator does not model them explicitly.
Third, this tool deliberately ignores architecture detail: layer count, KV head count, embedding sharing and tokenizer size all vary between model families, so two 7B models can differ by several hundred megabytes. Leave 10–15% headroom over the reported figure, and verify with your runtime's own memory report before you trust any number for a production decision. For architecture diagrams of your planned rig, Mermaid to SVG renders them cleanly.
Frequently asked questions
How much VRAM do I need to run a 7B model at Q4_K_M?
A 7B model at Q4_K_M needs about 4.2 GB for the weights alone, since Q4_K_M averages 4.8 bits per weight. With a 4,096-token context and runtime overhead, the LLM VRAM Calculator reports roughly 5–6 GB total, so it fits an 8 GB card with a little headroom.
Is the LLM VRAM Calculator free, and does it upload my inputs?
Yes, it is free and needs no account. All arithmetic runs in your browser, so the parameter count, quantization and context length you type are never sent to a server. Once the page has loaded it also works offline, which makes it convenient on a laptop you are using to configure a local inference box.
Does quantizing to Q2_K let me run larger models?
Quantization shrinks the weight memory directly: 70B at Q2_K is 70 × 3 ÷ 8, about 26 GB, versus 42 GB at Q4_K_M. So yes, a lower setting can bring a model inside your card's capacity. The trade-off is measurable quality loss, and 2-bit models often degrade badly on reasoning and code tasks.
Can a single RTX 4090 run a 70B model?
No. A 70B model at Q4_K_M is about 42 GB of weights, well beyond the 24 GB on an RTX 4090, before any context cache is added. The LLM VRAM Calculator's verdict will point you to two 24 GB cards with tensor parallelism, an A100 40GB-class GPU, or partial CPU offload at a large speed cost.
Why is my real VRAM usage higher than the estimate?
The calculator models weights, KV cache and a generic overhead, but not your desktop compositor, browser, batch size, or backend workspaces such as the CUDA context. Real usage on a 12 GB card can run a gigabyte or more above the estimate. Leave 10–15% headroom and confirm with your runtime's memory report.