gemma-4-E4B-it
Google- 8B parameters
- Dense
- Apache-2.0
- Released in bf16
- Estimated
google/gemma-4-E4B-it on Hugging Face
gemma-4-E4B-it with Glyd
| bf16 | Glyd | Change, % | ||
|---|---|---|---|---|
| Weights in GPU memory, GBEstimated from the parameter count | 15.99 | 10.77est. | −33 | |
| Smallest single GPU, GBWeights, an 8K-token KV cache and 1.5 GB for the runtime | 24 | 16 | ||
| H100 80 GB GPUs it takes | 1 | 1 |
Which single GPU it fits
Estimated from the parameter count.
| Memory | GPUs | bf16 | Glyd |
|---|---|---|---|
| 16 GB | RTX 4080, RTX 5080 | No | Fits |
| 24 GB | RTX 4090, RTX 3090, A10 | Fits | Fits |
| 32 GB | RTX 5090 | Fits | Fits |
| 48 GB | RTX A6000, L40S, RTX 6000 Ada | Fits | Fits |
| 80 GB | H100, A100 80 GB | Fits | Fits |
| 96 GB | RTX PRO 6000, GH200 | Fits | Fits |
| 141 GB | H200 | Fits | Fits |
Run it
pip install "glyd[gpu]"import glyd
model = glyd.from_pretrained("google/gemma-4-E4B-it")Linux and an NVIDIA GPU, Ampere or later: the weights are packed on the GPU as the model loads, every one bit for bit (getting started). vLLM serves Glyd's weights too, for the models checked there (serving with vLLM).
Related models
Measured so far
Twenty-two open models measured, with quality wherever bf16 fits one GPU.