Gemma VRAM Requirements: Gemma 4 and Gemma 3 Compared
Gemma VRAM requirements are low for the quality you get, because every Gemma model since Gemma 3 keeps most of its attention layers on a short sliding window. At 4-bit, one user at an 8K-token context needs about 9 GB for Gemma 4 12B, 19 GB for Gemma 4 26B-A4B and 22 GB for Gemma 4 31B. The older Gemma 3 27B needs about 19 GB. All of them fit on one 24 GB card at short context, and the 12B models fit on a 12 GB card.
Google released Gemma 4 in April 2026, so it is the generation to plan around for new work. Gemma 3 is still widely deployed, so both are covered below. The last section shows how to check any Gemma release yourself.
Memory by model
Weights are the approximate Q4_K_M and Q8_0 GGUF file sizes. Totals add the KV cache for one user at 8K tokens and 10% for working buffers, the method from our GPU memory for LLM guide:
| Model | Parameters | Q4_K_M weights | Q8_0 weights | Total, Q4 | Total, Q8 |
|---|---|---|---|---|---|
| Gemma 4 E2B | 2.3B effective (5.1B with embeddings) | ~3.1 GB | ~5.1 GB | ~3.5 GB | ~5.6 GB |
| Gemma 4 E4B | 4.5B effective (8B with embeddings) | ~5 GB | ~8.2 GB | ~5.6 GB | ~9.2 GB |
| Gemma 4 12B | 12B | ~7.4 GB | ~12.7 GB | ~8.6 GB | ~14.5 GB |
| Gemma 4 26B-A4B | 25.2B total, 3.8B active | ~17 GB | ~26.9 GB | ~19 GB | ~30 GB |
| Gemma 4 31B | 30.7B | ~18.3 GB | ~32.6 GB | ~22 GB | ~38 GB |
| Gemma 3 1B | 1B | ~0.8 GB | ~1.1 GB | ~0.9 GB | ~1.2 GB |
| Gemma 3 4B | 4B | ~2.5 GB | ~4.1 GB | ~3.1 GB | ~4.9 GB |
| Gemma 3 12B | 12B | ~7.3 GB | ~12.5 GB | ~9 GB | ~14.7 GB |
| Gemma 3 27B | 27B | ~16.6 GB | ~28.7 GB | ~19.4 GB | ~33 GB |
Parameter counts are from the Gemma 4 model cards. File sizes are from the GGUF builds by Unsloth and, for Gemma 4 12B, LM Studio. Different uploaders vary by a few hundred megabytes.
Image input needs one more file, the vision projector (mmproj). Add about 1 GB for most models, or about 0.2 GB for Gemma 4 12B, which projects image patches straight into the model instead of using a separate encoder. Gemma 3 1B is text-only.
Why the KV cache stays small
Gemma models interleave two kinds of attention layer. Local layers only look at a sliding window of recent tokens (512 tokens on the smallest models, 1,024 on the others), so their cache stops growing once the window is full. Global layers attend to the whole context, and only they grow with it. Gemma 3 uses five local layers for every global one; Gemma 4 uses a similar ratio and, on its larger models, gives the global layers few key-value heads.
The result is a cache per token that is a fraction of a dense model's:
| Model | Grows per token | Fixed (sliding layers) | 32K context | 128K context |
|---|---|---|---|---|
| Gemma 4 12B | 16 KB | ~0.34 GB | ~0.9 GB | ~2.5 GB |
| Gemma 4 26B-A4B | 20 KB | ~0.21 GB | ~0.9 GB | ~2.9 GB |
| Gemma 4 31B | 80 KB | ~0.84 GB | ~3.5 GB | ~11.6 GB |
| Gemma 3 4B | 20 KB | ~0.12 GB | ~0.8 GB | ~2.8 GB |
| Gemma 3 12B | 64 KB | ~0.34 GB | ~2.5 GB | ~8.9 GB |
| Gemma 3 27B | 80 KB | ~0.44 GB | ~3.1 GB | ~11.2 GB |
Gemma 4 E2B and E4B go further: their last 20 and 18 layers reuse the cache of earlier layers instead of storing their own, leaving about 6 KB and 16 KB per token. For them, the weights are nearly the whole memory bill.
Two caveats. First, Google's model card says the Gemma 4 global layers use "unified Keys and Values" to save memory. The Hugging Face Transformers implementation still caches keys and values separately, so the figures above assume both are stored; an engine that exploits the sharing will use less. Second, all of this depends on your inference engine allocating the sliding layers at their window size. If it reserves full-length cache for every layer, the numbers change a lot: Gemma 3 27B would need 496 KB per token, or about 67 GB at 128K instead of 11 GB. llama.cpp uses the reduced cache by default (its --swa-full flag turns that off, per the server README). Check what your engine does by watching memory as you raise the context limit.
Context length
Gemma 4 E2B and E4B support 128K tokens; Gemma 4 12B, 26B-A4B and 31B support 256K. Gemma 3 supports 128K, except the 1B model at 32K. At the full 256K, Gemma 4 31B needs about 22 GB of cache for one user, more than its 4-bit weights. Gemma 4 26B-A4B needs under 6 GB at 256K, which makes it the better choice for long documents on limited memory.
Why 26B-A4B needs 26B of memory
Gemma 4 26B-A4B is a mixture-of-experts model: 25.2 billion parameters in total, 3.8 billion active for each token. The active count sets the speed. The total sets the memory, because every expert stays loaded. So it generates tokens at about the pace of a 4B model while needing the memory of a 26B one. If speed is the constraint and you have the memory, it is the better pick over 31B. If memory is the constraint, Gemma 4 12B uses less than half as much. The Qwen VRAM requirements post covers the same trade-off for Qwen3 30B-A3B.
QAT checkpoints
Google publishes quantization-aware trained (QAT) versions of Gemma, trained so the model holds up better at 4-bit than one quantized after training. For Gemma 3, Google reported that QAT cut the 27B weights from 54 GB in BF16 to 14.1 GB in int4 while reducing the quality drop compared with ordinary post-training quantization, according to its QAT announcement. Official Q4_0 GGUF files exist for both generations, for example Gemma 4 31B QAT (17.7 GB) and Gemma 3 27B QAT (17.2 GB).
If you run Gemma at 4-bit, start from the QAT file rather than a standard Q4_K_M. The file sizes are similar; the QAT version was trained to be run that way. Our LLM quantization guide explains the levels and when to go higher.
Which GPUs fit Gemma
- 8 GB card: Gemma 4 E2B or E4B at Q4 with long context, Gemma 3 4B at Q8.
- 12 GB card: Gemma 4 12B or Gemma 3 12B at Q4, with room for a 32K context or several users.
- 16 GB card: the 12B models at Q8.
- 24 GB card (RTX 4090, RTX 3090, L4, A10): Gemma 4 31B or Gemma 3 27B at Q4 for one or two users at short context; Gemma 4 26B-A4B at Q4 with a 128K context.
- 48 GB card (L40S, RTX 6000 Ada, A6000): Gemma 4 31B at Q8 with a 64K context, or at Q4 with many users.
- 80 GB card (H100, A100 80GB): Gemma 4 31B at Q8 with the full 256K window, about 60 GB in total.
Licenses changed between generations
Gemma 4 is released under Apache 2.0, a standard open-source license that allows commercial use without extra terms. Gemma 3 and earlier use the Gemma Terms of Use, which allow commercial use but bind you to Google's prohibited use policy and require you to pass those restrictions on when you distribute the model or a derivative. If your legal team found the old terms a blocker, Gemma 4 removes that issue. The open-source LLM for enterprise post has a short license checklist.
Check any Gemma model yourself
The numbers above come from each model's config.json. Gemma 4 nests the text settings under text_config and gives global layers their own head size and key-value head count:
curl -s https://huggingface.co/google/gemma-4-31B-it/raw/main/config.json | python3 -c "
import json,sys; t=json.load(sys.stdin)['text_config']
print(t['layer_types'].count('full_attention'), 'global layers,',
t['num_global_key_value_heads'], 'global KV heads, head size', t['global_head_dim'])"
KV bytes per token = 2 x global layers x KV heads x head size x 2 (bytes at 16-bit)
Gemma 4 31B: 2 x 10 x 4 x 512 x 2 = 81,920 bytes (80 KB)
The Gemma 3 repositories on Hugging Face are gated, so you need to accept the license and sign in to read their configs.
Which plan size a Gemma model falls into
AI Refinery, a self-hosted LLM platform, sizes its plans by parameter count: Basic covers models up to 14B, Professional up to 70B and Enterprise has no limit. In those terms Gemma 4 E2B, E4B and 12B and Gemma 3 up to 12B fit Basic, which includes a 30-day free trial. Gemma 4 26B-A4B and 31B and Gemma 3 27B fall under Professional. Which models are in the catalog at any time is shown in the console, not in this post. Plans are in the pricing section.
Each catalog variant lists measured GPU memory with the KV cache counted at the full context window, so expect the console figure to sit nearer the long-context numbers than the 8K ones. The same sizing applies to other families in Llama 70B VRAM requirements. For the platform itself, see the private model catalog overview.
Figures are estimates from public configs and file sizes, checked in November 2026. Measure on your own hardware before buying GPUs.