VRAM · Local model
Gemma 4 31B VRAM requirements
Gemma 4 31B has 31.3 billion parameters. With an 8K context it needs about 21 GB of memory at Q4_K_M, which fits a 24 GB GPU such as the GeForce RTX 3090, and 66 GB at full precision.
- Parameters
- 31.3B
- Layers
- 60
- Max context
- 262,144
- Experts
- Dense
Attention: 10 full-attention layers, 50 sliding-window layers (last 1,024 tokens).
Requirements
VRAM by quantisation and context length
| Quantisation | 4K context | 32K context | 128K context | 256K context |
|---|---|---|---|---|
| FP16 / BF16 | 65 GB | 68 GB | 76 GB | 87 GB |
| FP8 | 33 GB | 36 GB | 44 GB | 55 GB |
| Q8_0 (8-bit) | 35 GB | 38 GB | 46 GB | 57 GB |
| Q6_K | 27 GB | 30 GB | 38 GB | 49 GB |
| Q5_K_M | 24 GB | 26 GB | 35 GB | 46 GB |
| Q4_K_M | 21 GB | 23 GB | 31 GB | 42 GB |
| MXFP4 | 18 GB | 21 GB | 29 GB | 40 GB |
| Q3_K_M | 17 GB | 20 GB | 28 GB | 39 GB |
| Q2_K | 14 GB | 16 GB | 25 GB | 36 GB |
Weights + FP16 KV cache + overhead (10%, at least 1 GB), batch 1. See the formula and assumptions.
Hardware
What can run it at Q4_K_M
On one GPU: GeForce RTX 3090, GeForce RTX 4090, Radeon RX 7900 XTX, GeForce RTX 5090, L4, L40S, A100 80GB, H100 80GB, RTX PRO 6000 Blackwell, H200, Instinct MI300X. Split across consumer cards: 3 × GeForce RTX 4060, 2 × GeForce RTX 3060, 2 × GeForce RTX 4070, 2 × GeForce RTX 5070. On a Mac: 32 GB of unified memory or more.
Calculator
Try other settings
31.3B parameters · 262,144-token context · model card
GGUF; the most popular balance of size and quality.
Requests served at the same time.
- Weights
- 18 GB
- KV cache
- 1.4 GB
- Overhead
- 1.9 GB
- Some layers only look at the last 1,024 tokens, so the cache grows more slowly with context.
| GPU | Memory | Runs it? |
|---|---|---|
| GeForce RTX 4060 | 8 GB | With 3 GPUs |
| GeForce RTX 3060 | 12 GB | With 2 GPUs |
| GeForce RTX 4070 | 12 GB | With 2 GPUs |
| GeForce RTX 5070 | 12 GB | With 2 GPUs |
| GeForce RTX 4060 Ti 16GB | 16 GB | With 2 GPUs |
| GeForce RTX 4080 Super | 16 GB | With 2 GPUs |
| GeForce RTX 5060 Ti 16GB | 16 GB | With 2 GPUs |
| GeForce RTX 5070 Ti | 16 GB | With 2 GPUs |
| GeForce RTX 5080 | 16 GB | With 2 GPUs |
| GeForce RTX 3090 | 24 GB | Yes |
| GeForce RTX 4090 | 24 GB | Yes |
| Radeon RX 7900 XTX | 24 GB | Yes |
| GeForce RTX 5090 | 32 GB | Yes |
| L4 | 24 GB | Yes |
| L40S | 48 GB | Yes |
| A100 80GB | 80 GB | Yes |
| H100 80GB | 80 GB | Yes |
| RTX PRO 6000 Blackwell | 96 GB | Yes |
| H200 | 141 GB | Yes |
| Instinct MI300X | 192 GB | Yes |
Apple Silicon Macs (unified memory)
- 16 GB: doesn’t fit
- 24 GB: doesn’t fit
- 32 GB: fits
- 36 GB: fits
- 48 GB: fits
- 64 GB: fits
- 96 GB: fits
- 128 GB: fits
- 192 GB: fits
- 256 GB: fits
- 512 GB: fits
Filled: fits as is. Outlined: fits after raising the GPU memory limit (leaving 8 GB for macOS). macOS gives the GPU two-thirds of memory up to 32 GB and three-quarters above by default.
FAQ
Frequently asked questions
How much VRAM does Gemma 4 31B need?
With an 8K context, about 21 GB at Q4_K_M, 36 GB at 8-bit (Q8_0) and 66 GB at full 16-bit precision. That covers the weights, the KV cache and runtime overhead.
Can Gemma 4 31B run on a 24 GB GPU?
Yes, at Q4_K_M or smaller, with an 8K context (about 21 GB). Longer contexts need more memory for the KV cache.
Can I run Gemma 4 31B on a Mac?
Yes: at Q4_K_M and an 8K context it fits a Mac with 32 GB of unified memory using macOS’s default GPU memory limit.
How much memory does Gemma 4 31B’s context use?
Its KV cache grows by about 78 MB for every 1,000 tokens of context in FP16 (its sliding-window or chunked layers stop growing once their window is full). At its full 262,144-token context the cache is about 21 GB. Quantising the cache to 8-bit roughly halves it.
Related