Try adjusting your filters or search query
This tool never shows a number without saying how confident it is. A figure with no symbol next to it was computed from the model's own published config; a figure with a symbol was computed from something less direct, and the symbol tells you exactly what.
Every VRAM and KV-cache figure here is arithmetic — parameter count × bytes per parameter, attention geometry × tokens — not a reading from a real vLLM process. Use them to shortlist hardware, then confirm by actually starting the server.
The model's own HuggingFace repo is gated, so its attention geometry (layers, KV heads, head dim, sliding window) was read from an ungated repo of the same architecture. Almost always right — but it is a stand-in, not the model's own file.
No config.json was reachable anywhere — gated repo, no mirror. The
geometry was entered by hand from the model card or paper. Treat the KV number as
a rough guide and verify before you size hardware around it.
The weakest number on the site. With no attention geometry at all, the KV cache is approximated straight from parameter count and context length. It ignores GQA and MLA (which shrink the cache a lot) and over-estimates MoE models, so it usually reads too high. Don't rule a model out on this figure alone.
The per-GPU headroom set aside for activations and CUDA graphs (the Reserve control in the filter bar) is a guess, not a calculation. vLLM measures it at startup by profiling a real forward pass, so it shifts with batch size, sequence length and model. Raise it to be safe, lower it to see what a tighter server would fit.
The quantized weights load and run on this GPU, but the GPU has no tensor cores for that format — vLLM dequantizes on the fly (e.g. INT4/AWQ via Marlin). You get the memory saving, not a speedup, and throughput can be worse than a natively-supported format. A plain ✓ means native hardware support.
✓ = model weights (plus the KV cache, if you switched the KV estimate on) sit inside the usable budget: physical VRAM × memory utilization − activation reserve. ✗ = too large, or the quantization format isn't supported on that GPU at all. Hover any verdict for the arithmetic behind it.
A card's bar splits the selected GPUs' usable memory into weights, KV cache and free space — red when it overflows. It is drawn from the same fit result that decides the ✓/✗, so the bar and the verdict can never disagree.
Some or all of the model's layers cap their KV cache at a fixed window instead of growing with the full context, so its KV stays far smaller than a full-attention model of the same size. A surprisingly small KV number on an SWA model is the design, not a bug.
Diffusion-style and other single-pass models have no autoregressive KV cache at all — context length costs them no extra VRAM. Their KV row reads zero on purpose.
Want the hardware side — which quantization formats run natively on which GPU, and how the memory budget is derived? Open .