Qwen3-8B
Qwen- 8.2B parameters
- Dense
- Apache-2.0
- Released in bf16
- Measured Sep 26, 2026
Qwen/Qwen3-8B on Hugging Face
Qwen3-8B with Glyd
| bf16 | Glyd | Change, % | ||
|---|---|---|---|---|
| Weights in GPU memory, GBEvery weight restored bit for bitlog | 16.38 | 11.15 | −32 | log |
| Smallest single GPU, GBWeights, an 8K-token KV cache and 1.5 GB for the runtime | 24 | 16 | ||
| H100 80 GB GPUs it takes | 1 | 1 | ||
| Its matrices, GBPacked and unpacked, every one bit for bitlog | 13.9 | 9.4 | −32 | log |
| MMLU, 300 questions, %Answers as bf16's on 99.67%log | 74.0 | 74.3 | log | |
| Perplexity, enwik8log | 20.7490 | 20.7428 | log |
Which single GPU it fits
Worked out from the measured weights.
| Memory | GPUs | bf16 | Glyd |
|---|---|---|---|
| 16 GB | RTX 4080, RTX 5080 | No | Fits |
| 24 GB | RTX 4090, RTX 3090, A10 | Fits | Fits |
| 32 GB | RTX 5090 | Fits | Fits |
| 48 GB | RTX A6000, L40S, RTX 6000 Ada | Fits | Fits |
| 80 GB | H100, A100 80 GB | Fits | Fits |
| 96 GB | RTX PRO 6000, GH200 | Fits | Fits |
| 141 GB | H200 | Fits | Fits |
Against DFloat11, RTX 4080 SUPER
Higher is better. Both hold the weights in about 11.2 GB.
| Tokens/s | |||
|---|---|---|---|
| Sequences a step | DFloat11 | Glyd | Glyd against DFloat11, × |
| 1 | 13.8 | 47.2 | 3.4 |
| 8 | 104.7 | 360.2 | 3.4 |
| 32 | 387.5 | 1,268.6 | 3.3 |
| 64 | 746.2 | 1,897.2 | 2.5 |
Run it
pip install "glyd[gpu]"import glyd
model = glyd.from_pretrained("Qwen/Qwen3-8B")Linux and an NVIDIA GPU, Ampere or later: the weights are packed on the GPU as the model loads, every one bit for bit (getting started). vLLM serves Glyd's weights too, for the models checked there (serving with vLLM).
Related models
Also measured
Twenty-one more open models measured, with quality wherever bf16 fits one GPU.