One total, split into its parts
Weights from parameters and bits per weight, KV cache from layers, context and batch, plus an activation allowance. You see the three parts and their share, not just a number.
Local VRAM · GGUF quantization · API batch cost
PromptMeter answers two questions before you download a GGUF file or send an API batch: how much VRAM the model needs for its weights, KV cache and activation overhead at your quantization, context length and batch size, and what one run of your batch costs on a dated price table you can edit. Both numbers come with the formula behind them, the reference date of the data, and a plain note of what the estimate cannot see. Everything is computed in this tab — no upload, no account, no server call.
Pick a model preset from 1.5B to 671B or type your own parameters, choose FP16 down to Q2_K, set 2K–128K of context, FP16/Q8/Q4 KV cache and a batch size, and the page splits the total VRAM into weights, KV cache and overhead, marks each GPU capacity from 8 GB to 80 GB as fits or short, counts the cards a multi-GPU build needs, and gives a bandwidth-based tokens-per-second estimate. The second tab does the API side: context utilisation, per-request and batch cost, wall-clock time and a 30-day projection, against a price table whose basis date is on screen and whose every cell you can change.
Total VRAM needed
0GiB
GPU fit at this setting
—
weights_GiB = P × bpw ÷ 8 × 0.9313
KV_GiB = kv_KiB/token ÷ 1,048,576 × tokens × batch
overhead_GiB = 0.40 + 0.10 × √P + 0.00025 × (tokens ÷ 1024) × batch
tok/s ≈ bandwidth × 0.55 ÷ weights_GiB
The per-token KV value is for this model's layer count and KV heads at FP16; the quantization factor and the batch size are applied on top.
Batch total cost
$0.00
—
cost_per_request = prompt ÷ 1M × price_in + output ÷ 1M × price_out
batch_cost = cost_per_request × requests
context_use = (prompt + output) ÷ context_window
latency ≈ 0.6 s + prompt ÷ prefill_tok/s + output ÷ output_tok/s
wall_clock = ⌈ requests ÷ concurrency ⌉ × latency
Prefill and decode rates are assumptions from the price table; real latency depends on provider load and on thinking tokens, which are billed as output.
Basis date 2026-11-21 · Provider defaults
| Model | Input $/1M | Output $/1M | Context | Prefill tok/s | Output tok/s |
|---|---|---|---|---|---|
| GPT-6.2 AstraOpenAI | |||||
| GPT-6.2 SolOpenAI | |||||
| GPT-6.1 TerraOpenAI | |||||
| Claude Fable 5.2Anthropic | |||||
| Claude Opus 5.5Anthropic | |||||
| Claude Sonnet 5.5Anthropic | |||||
| Claude Haiku 5.5Anthropic | |||||
| Gemini 4 CarbonGoogle | |||||
| Gemini 4 NeonGoogle | |||||
| Gemini 3.8 FlashGoogle | |||||
| DeepSeek FlashDeepSeek | |||||
| DeepSeek V4 ProDeepSeek |
Every cell can be edited: put in your own contract rate, a different context window or a measured decode speed. Edits stay in this browser only.
Promotional rates in this snapshot: Gemini 4 Carbon is 2/10 per million until 2026-11-30 and 4/20 from 2026-12-01; Gemini 3.8 Flash is 0.75/3.75 until 2026-12-31 and 1.50/7.50 from 2027-01-01; DeepSeek figures are peak-hour rates and off-peak is half; Claude Haiku 5.5 charges 0.50/2.50 above 100K prompt tokens.
What it does
Sizing on the left of the decision, spending on the right.
Weights from parameters and bits per weight, KV cache from layers, context and batch, plus an activation allowance. You see the three parts and their share, not just a number.
FP16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M and Q2_K, each with its approximate bits per weight, so the saving of one step down is visible before you download anything.
Context from 2K to 128K, KV cache in FP16, Q8 or Q4, and batch size applied to every token stored. Long context and batching multiply the same cache.
Each capacity from 8 GB to 80 GB is marked fits or short in GiB, with a safety reserve you can switch off, and the selected card's count is computed for multi-GPU builds.
A memory-bandwidth approximation for single-stream and batched decode, so the number is checkable rather than mysterious — and it says how far it can be off.
Context utilisation, per-request and batch cost, wall-clock and monthly projection, from a price table dated on screen; change any unit price and the result follows instantly.
How it works
No sign-up and no setup; every step happens on your device.
Choose a preset or select Custom and type the parameter count and KV size per token. Both fields are shown so nothing is hidden in a preset.
Quantization, context length, KV cache precision and batch size decide most of the total. The breakdown and the GPU verdicts update as you type.
The total, the three parts, the safety margin, the cards you need and the tokens/s estimate sit together, with the notes that apply to your current settings.
Switch to the API tab, set tokens and volume, adjust any unit price, then copy or download the whole configuration as JSON or Markdown.
Reference
Weights only, in GiB, before any KV cache or overhead. The calculator uses the same bits-per-weight values.
| Level | Bits / weight | 8B weights (GiB) | 70B weights (GiB) | 405B weights (GiB) |
|---|---|---|---|---|
FP1616-bit | 16.00 | 14.90 | 130.4 | 754 |
Q8_08-bit, near-lossless | 8.50 | 7.92 | 69.3 | 401 |
Q6_KK-quant | 6.56 | 6.11 | 53.5 | 309 |
Q5_K_MK-quant, medium | 5.69 | 5.30 | 46.4 | 268 |
Q4_K_MK-quant, medium | 4.85 | 4.52 | 39.5 | 229 |
Q3_K_MK-quant, medium | 3.91 | 3.64 | 31.9 | 184 |
Q2_Ksmallest, visibly lossy | 2.63 | 2.45 | 21.4 | 124 |
Bits per weight are the usual averages reported for llama.cpp K-quants and can differ slightly between files. Q2_K and Q3_K_M trade visible quality for size; Q4_K_M is the common default for a reason.
Reference
The last column applies the page's tokens/s formula to an 8B model at Q4_K_M (about 4.5 GiB of weights), single stream, before any KV cache.
| GPU | VRAM (GB) | Bandwidth (GB/s) | 8B Q4_K_M, ~tok/s |
|---|---|---|---|
| RTX 4060 Ti 16 GB | 16 | 288 | ~35 |
| RTX 4070 Ti SUPER 16 GB | 16 | 672 | ~82 |
| RTX 4080 SUPER 16 GB | 16 | 736 | ~90 |
| RTX 3090 24 GB | 24 | 936 | ~114 |
| RTX 4090 24 GB | 24 | 1008 | ~123 |
| RTX 5090 32 GB | 32 | 1792 | ~218 |
| RTX A6000 48 GB | 48 | 768 | ~94 |
| RTX 6000 Ada 48 GB | 48 | 960 | ~117 |
| L40S 48 GB | 48 | 864 | ~105 |
| A100 80 GB | 80 | 2039 | ~248 |
| H100 80 GB SXM | 80 | 3350 | ~408 |
| Apple M4 Max 128 GB (unified) | 128 | 546 | ~66 |
| Apple M2 Ultra 192 GB (unified) | 192 | 800 | ~97 |
Bandwidth figures are the manufacturers' peak memory bandwidth. Real decode throughput is usually 40–70% of the value this formula gives, depending on the runtime, the kernel and what else the GPU is doing.
Method
Every formula on this page is written out here, including the assumptions.
Weights are the easy part: parameters × bits per weight ÷ 8, converted from billions of bytes to GiB. A 70B model at Q4_K_M is 70.6 × 4.85 ÷ 8 = 42.8 billion bytes, or 39.9 GiB. Nothing about the context or the batch changes this number, which is why it is the floor of every build.
The KV cache is what people underestimate. It is two tensors (key and value) per layer, per token, per sequence, so its size grows with context length and batch size at the same time. For Llama 3.1 8B each token costs 128 KiB at FP16, which is 1 GiB at 8K context and 8 GiB at 64K. Quantizing the cache to Q8 roughly halves it, to Q4 roughly quarters it.
Activation overhead covers the compute buffers, logits and fragmentation a runtime needs beyond weights and cache. It depends on the kernel and the batch, so this page uses a simple stated allowance: 0.40 GiB plus 0.10 GiB per square root of the parameter count in billions, plus 0.00025 GiB per 1,024 tokens per batch slot. Treat it as a planning allowance.
The API side is arithmetic on published list prices: prompt tokens and completion tokens are priced separately per million, and the output side usually dominates. Timing is a simple queue model: a fixed first-token delay, prompt processing at an assumed prefill rate, output generation at an assumed decode rate, and requests grouped into waves by the concurrency you set. Providers' own latency varies with load, region and thinking tokens, so read the result as an order of magnitude, not a promise.
weights_GiB = P(B) × bpw ÷ 8 × 0.9313KV_GiB = kv_KiB_per_token × factor ÷ 1,048,576 × tokens × batchoverhead_GiB = 0.40 + 0.10 × √P + 0.00025 × (tokens ÷ 1024) × batchtotal_GiB = weights + KV + overheadtok/s ≈ bandwidth_GB/s × 0.55 ÷ weights_GiB; batch × batch^0.72per_request = prompt ÷ 1M × price_in + output ÷ 1M × price_outlatency ≈ 0.6 s + prompt ÷ prefill + output ÷ decode; wall = ⌈requests ÷ concurrency⌉ × latencyQuestions
About 4.5 GiB for the weights (8 × 4.85 bits per weight), plus the KV cache and overhead. For Llama 3.1 8B at 8K context the cache is 1 GiB, and the page's overhead allowance is about 0.7 GiB, so the plan lands near 6.2 GiB. An 8 GB card fits with about 0.5 GiB of headroom after the safety reserve; at 32K context the cache alone is 4 GiB and the total passes 9 GiB, so the same card no longer fits.
At Q4_K_M the weights alone are about 39.9 GiB, the KV cache adds 2.5 GiB at 8K context and the overhead allowance about 1.2 GiB, so roughly 43.6 GiB. A 48 GB card holds it, and so do two 24 GB cards once the safety reserve is applied; tensor parallelism then costs some effective bandwidth.
The calculator counts weights, KV cache and a fixed activation allowance. A real run also holds the CUDA context, the graphics driver and desktop, allocator fragmentation, and runtime-specific buffers; compiled kernels and long-context attention can add more. Keep 10–20% free, which is what the safety margin is for.
Roughly: Q8 about half and Q4 about a quarter of the FP16 cache, plus scale overhead. On Llama 3.1 8B at 64K context that turns 8 GiB into about 4.2 GiB or 2.2 GiB. It needs flash-attention style kernels and is not supported by every runtime, and quality can drop on long contexts, so measure before relying on it.
They are list prices read from the providers' public price pages on 2026-11-21, shown in the table with that date. Some are promotional: Gemini 4 Carbon is 2/10 per million until 2026-11-30 and 4/20 after, Gemini 3.8 Flash is 0.75/3.75 until 2026-12-31, and DeepSeek prices halve off-peak. Every cell is editable, including context windows, so you can put in your own contract rate.
It is a bandwidth-bound approximation: memory bandwidth × 0.55 ÷ weight size in GiB. For an 8B Q4_K_M model on a card with 1,000 GB/s it gives about 120 tok/s, which is in the range people report for that combination. Compute-bound models, very small batches, CPU offload and prompt processing can all miss it by much more than 40%, so treat it as a comparison between setups.
No. Both calculators, the price table and the export are JavaScript in this page; the tool makes no fetch, XHR or beacon call. Your settings are kept in your own browser's localStorage and the only external request on the page is the advertising loader in the head.
Honest limits
The VRAM total is a model, not a reading from your machine. Different runtimes, kernels and compiled attention paths use different amounts, and a configuration that the page marks as fitting can still fail to load.
Activation memory depends on the batch shape, the kernel and the context; the 0.40 + 0.10√P allowance is deliberately simple. Very long contexts can push attention buffers far above it.
The estimate assumes decode is limited by reading the weights from memory. A compute-bound model, a tiny batch, CPU offload or a busy desktop GPU breaks that assumption.
The table is a dated snapshot of list prices. Providers tier by context length, cache hits, region and batch processing, and reasoning tokens are billed as output; none of that is modelled here. Put your own rate into the table for a real budget.