VRAM guide
Running large language models locally comes down to one resource above all: video memory (VRAM). The model weights have to fit into your GPU's VRAM in full – along with a small overhead – or inference stutters, spills over into system RAM, or fails outright. Quantization shrinks the footprint: a 4-bit (int4) model needs roughly a quarter of the fp16 size, 8-bit (int8) about half. This page translates the VRAM requirements of popular open-source LLMs into concrete GPU recommendations – with current European shop prices and hourly hosting rates side by side, so you can weigh a one-time purchase against renting the same card in the cloud. The fit figures use the same heuristic as our GPU detail pages: estimated VRAM need = parameters × factor + 1.5 GB, with factors 2.0 (fp16), 1.0 (int8) and 0.5 (int4). Budget extra headroom for long contexts and the KV cache – treat the numbers as a solid starting point, not a hard limit.
Model weights must fit into VRAM in full, plus a small overhead. Quantization shrinks the footprint: int4 needs roughly a quarter of the fp16 size, int8 about half. The table shows the largest models (in billions of parameters) that fit each VRAM budget.
| VRAM budget | fp16 (16-bit) | int8 (8-bit) | int4 (4-bit) |
|---|---|---|---|
| 8 GB | ≈3.2 B | ≈6.5 B | ≈13.0 B |
| 16 GB | ≈7.2 B | ≈14.5 B | ≈29.0 B |
| 24 GB | ≈11.2 B | ≈22.5 B | ≈45.0 B |
| 48 GB | ≈23.2 B | ≈46.5 B | ≈93.0 B |
| 96 GB | ≈47.2 B | ≈94.5 B | ≈189.0 B |
Estimated VRAM need = parameters × factor + 1.5 GB, with factors 2.0 (fp16), 1.0 (int8) and 0.5 (int4) – the same heuristic as the fit table on every GPU detail page.
Graphics cards with current shop offers, grouped by VRAM – each tier lists the open-source LLMs that fit (int4).
Open-source LLMs that fit 288 GB (int4): Qwen3-Coder-Next (0 B)Gemma 3 1B PT (1 B)Qwen3.5-2B (2 B)
flagship
Open-source LLMs that fit 192 GB (int4): Qwen3-Coder-Next (0 B)Gemma 3 1B PT (1 B)Qwen3.5-2B (2 B)
flagship
Open-source LLMs that fit 141 GB (int4): Qwen3-Coder-Next (0 B)Gemma 3 1B PT (1 B)Qwen3.5-2B (2 B)
flagship
flagship
Open-source LLMs that fit 96 GB (int4): Qwen3-Coder-Next (0 B)Gemma 3 1B PT (1 B)Qwen3.5-2B (2 B)
Open-source LLMs that fit 80 GB (int4): Qwen3-Coder-Next (0 B)Gemma 3 1B PT (1 B)Qwen3.5-2B (2 B)
high
flagship
flagship
Open-source LLMs that fit 48 GB (int4): Qwen3-Coder-Next (0 B)Gemma 3 1B PT (1 B)Qwen3.5-2B (2 B)
mid
high
high
high
No – not in any common precision. A 70B model needs roughly 141 GB at fp16 and still about 36.5 GB at int4, so 24 GB falls short. You need a 48 GB card (or a dual-GPU setup) for 70B models at 4-bit.
It depends on the task. For chat, summarization and coding, a good 4-bit quantization is usually close to fp16; subtle degradation shows up mainly in long reasoning chains, math and low-resource languages. The fastest check is trying both precisions on your own prompts.
About 17.5 GB at fp16, 9.5 GB at int8 and 5.5 GB at int4 (weights plus a small overhead). An 8 GB card runs 8B models comfortably at 4-bit; for int8 you should plan 12 GB or more.
Only a small fixed overhead of 1.5 GB. Long contexts, large batches and vision adapters add several more gigabytes – for 32k+ contexts, budget roughly 10–25 % extra VRAM on top of the model size.
For starters, a 12–16 GB card covers 7B–14B models at int4 and 8B models even at int8. If you only experiment occasionally, renting the same GPU by the hour often beats the purchase price – compare current shop prices and hosting rates on this page.