Qwen3.8 27B
Qwen/Qwen3.8-27B on Hugging Face
Weights in GPU memory
55.6 to 37.3 GB
33% less, every weight restored bit for bitSmallest single GPU
80 GB to 48 GB
Weights, an 8K-token KV cache and 1.5 GB for the runtimeH100 80 GB GPUs it takes
1 to 1
Weights, an 8K-token KV cache and 1.5 GB a GPU for the runtimeMMLU, 300 questions
79.7% to 80.0%
Same weights; the products sum in another orderWhich single GPU it fits
Worked out from the measured weights.
| Memory | GPUs | bf16 | Glyd |
|---|---|---|---|
| 16 GB | RTX 4080, RTX 5080 | No | No |
| 24 GB | RTX 4090, RTX 3090, A10 | No | No |
| 32 GB | RTX 5090 | No | No |
| 48 GB | RTX A6000, L40S, RTX 6000 AdaRoom for about 192K tokens of context with Glyd | No | Fits |
| 80 GB | H100, A100 80 GBRoom for the KV cache: about 433K tokens in bf16, 711K with Glyd | Fits | Fits |
| 96 GB | RTX PRO 6000, GH200 | Fits | Fits |
| 141 GB | H200 | Fits | Fits |
Every matrix, bit for bit
Every Linear layer's matrix packed and unpacked; A10, Sep 27, 2026.
| Matrices in bf16 | 49.5 GB |
|---|---|
| With Glyd, tiered layout | 33.3 GB |
| Change | −32.8% |
The 12-bit layout takes 24.7 to 24.8% off each matrix and decodes with less work. The ten-model report
Same weights, same model
Every weight decodes to exactly the bf16 value it was. The matrix products add their terms in a different order than cuBLAS does, as any two GPU kernels do, so a few close answers can flip either way.
| Benchmark | bf16 | Glyd |
|---|---|---|
| MMLU, 300 questions | 79.7% | 80.0% |
| Perplexity, enwik8 | 15.1946 | 15.1941 |
H100 SXM, Sep 27, 2026. Lower perplexity is better.
Run it
git clone https://github.com/surya-koritala/Glyd
cd Glyd/gpu
python e2e.py /models/Qwen3.8-27B --format auto --fused \
--merge --baseline --batch 1,8,32Today Glyd runs from its PyTorch harness; vLLM and SGLang integration is not built yet.
Also measured
Twenty more open models measured, with quality wherever bf16 fits one GPU.