LLM VRAM Calculator: How Much VRAM Does a Local Model Really Need?
Reviewed by the OnlineFree.app team · Updated
Key points
- A 7B model at Q4_K_M needs about 4.2 GB of weights, and total VRAM is always higher.
- Weight memory equals parameters in billions times bits per weight, divided by eight.
- Quantization uses more bits than its label suggests: Q4_K_M stores 4.8 bits per weight, not 4.
- KV cache grows linearly with context, so 32K context costs eight times the KV memory of 4K.
- The LLM VRAM Calculator gives a fit verdict across 8, 12, 16 and 24 GB tiers without a sign-up.
How much VRAM does a local LLM need?
A 7B model at Q4_K_M needs roughly 4.2 GB for its weights alone, and total VRAM lands above that once KV cache and runtime overhead are added. An 8 GB card handles that combination at a 4K context comfortably; at 32K the same model starts to get tight.
Memory for local inference splits into three parts. Weights are the model file itself, fixed by parameter count and quantization. The KV cache stores attention keys and values for every token currently in context, so it grows with whatever you feed in. Runtime overhead covers the CUDA context, memory fragmentation and scratch buffers the inference engine allocates.
The LLM VRAM Calculator takes those three inputs — parameter count in billions, weight quantization, context length — and returns a single figure in GB to one decimal, plus a compact breakdown showing how much each part contributed. The breakdown matters more than the headline number, because it tells you which knob to turn when the model does not fit.
What sets LLM VRAM Calculator apart
Most sizing resources answer a slightly different question. Model cards list file sizes but say nothing about KV cache. Vendor inference documentation assumes batched datacenter workloads with paged attention and multi-GPU sharding. Forum threads give you an answer for one card, one model, one context length.
The most accurate alternative is loading the model and reading the memory line after llama.cpp or Ollama reports it — reliable, but useless when you are still deciding which GPU to buy. The LLM VRAM Calculator sits in that gap: three inputs, an instant answer, and a plain-language verdict that names the GPU tiers it fits instead of leaving you to compare numbers yourself.
It also runs entirely in the browser. No account, no upload, no server round-trip, and it keeps working if you save the page and go offline on the laptop you are using to price hardware. If you want the same instant-answer approach for other jobs, the wider collection is at OnlineFree.app.
How do parameters become gigabytes?
Weight memory is simple arithmetic: parameters in billions multiplied by bits per weight, divided by eight. A 7B model at FP16 is 7 × 16 ÷ 8 = 14 GB. The same model at Q4_K_M is 7 × 4.8 ÷ 8 = 4.2 GB. Memorize that one line and you can sanity-check any estimator, including ours.
The bits-per-weight figures are deliberately not round numbers. Q4_K_M uses 4.8 bits rather than 4.0 because k-quant blocks store scale and minimum values alongside the weights, and a handful of sensitive tensors stay at higher precision. Q8_0 is 8.5 bits for the same reason. Plugging a flat 4.0 into the formula understates a real Q4_K_M file by roughly 17 percent.
The full ladder for a 7B model: FP16 14.0 GB, Q8_0 7.4 GB, Q6_K 5.8 GB, Q5_K_M 5.0 GB, Q4_K_M 4.2 GB, Q3_K_M 3.4 GB, Q2_K 2.6 GB. Scale it up and the reason 70B builds are expensive becomes obvious — 70B at Q4_K_M is 42 GB of weights before a single token of context, which pushes you straight past 24 GB cards.
Why context length changes the answer
KV cache memory scales linearly with the number of tokens in context. Moving from 4,096 to 32,768 tokens multiplies that portion by eight; 131,072 multiplies it by thirty-two, with no change to the weights at all. That single relationship explains most of the disagreement between people reporting memory use for the same model file.
It also explains why long-context workloads surprise people. A 7B model that idles on an 8 GB card during short chat sessions can refuse to load when an agent pastes in a large document, because the cache — not the weights — grew past the limit.
The estimator treats context length as a first-class input rather than a footnote. Set it to the length you genuinely use: 4K is honest for chat, while RAG pipelines and coding agents that stuff documents into the prompt belong at 32K or higher. When unsure, run both and plan for the larger figure.
What the GPU fit verdict really means
The output pairs the GB figure with a verdict against the common tiers — 8, 12, 16 and 24 GB — and then names what comes next. A 24 GB RTX 4090 or 3090 is the practical ceiling for single-card consumer builds; beyond that you are looking at an RTX A6000 48 GB, an A100 40GB or 80GB, or splitting the model across two cards with tensor parallelism.
Two caveats are worth stating plainly. The estimate assumes a single sequence, because there is no batch size input — serving several concurrent users needs headroom above the number shown. And tensor parallelism is close to linear for weights but not free: communication buffers and duplicated activations cost extra, so 2 × 24 GB delivers somewhat less than 48 GB of usable model space.
Then subtract what your desktop already holds. A compositor, a browser and a display can occupy a gigabyte or more of VRAM, which is often the difference between a 7B model fitting on an 8 GB card and silently spilling to system RAM. After loading, confirm real usage with nvidia-smi or your inference server's memory line; treat the calculator as a planning tool, not a guarantee.
Frequently asked questions
Is the LLM VRAM Calculator free, and does it send my inputs anywhere?
Yes, it is free and nothing leaves your browser. The LLM VRAM Calculator runs entirely client-side — the arithmetic happens in JavaScript on your machine, with no sign-up, no usage limit and no server round-trip recording your model choices. Once the page is cached it keeps working offline.
Does the LLM VRAM Calculator account for batch size or tensor parallelism?
It assumes one sequence and has no batch size input. Batching multiplies KV cache memory, so serving several users at once needs more than the figure shown. Tensor parallelism appears in the verdict for split setups, but real multi-GPU runs also pay communication buffers, so 2 × 24 GB is approximately, not exactly, 48 GB.
Can a 24 GB GPU run a 70B model?
Not at usable speed without offloading. A 70B model at Q4_K_M needs about 42 GB of weights before any context is added, which exceeds 24 GB by a wide margin. You would need an A100 40GB or larger, two 24 GB cards with tensor parallelism, or partial CPU offload that cuts tokens per second sharply.
Which quantization should I choose for an 8 GB card?
Q4_K_M is the usual starting point: a 7B model lands near 4.2 GB of weights, leaving room for context and overhead inside 8 GB. Q5_K_M adds roughly 0.8 GB and is worth it if you stay under 4K context. Go to Q3_K_M only when you must — quality loss becomes noticeable below 4 bits.
How accurate is the VRAM estimate?
It is a planning estimate, not a measurement. Weight memory is pure arithmetic and dependable; KV cache and runtime overhead are modelled, so real usage can differ by a gigabyte or more depending on the inference engine, flash attention and KV cache quantization. Load the model and check nvidia-smi before buying hardware on one number.