Report · Sep 27, 2026 · Glyd
Nineteen open models: every matrix 32 to 33% smaller, bit for bit
Every Linear layer's matrix of nineteen open models, SmolLM3 3B to GLM-4.5-Air, packed and unpacked bit for bit: 31.9 to 33.0% smaller, quality on ten.
Ten popular open models were measured first; nine more have been since, from GLM-4.5-Air and Llama 4 Scout down to Llama 3.2 3B. On all nineteen, Glyd’s tiered layout stores every Linear layer’s matrix 31.9 to 33.0% smaller than bf16, and every matrix unpacks to exactly the bf16 values it held. Where quality was measured, on ten of them, bf16 and Glyd score the same to noise.
The sizes
sizes.py packs every Linear layer’s matrix of a model, unpacks it and compares the result with the original bit for bit. Two things it counts that a quick sum would miss: a layer’s experts kept as one tensor (Gemma 4, Llama 4) count as a matrix an expert, and every projection is counted, the linear-attention inputs of Qwen3-Next and Qwen3.8 included.1
| Model | Matrices in bf16 | Glyd | Change | Run |
|---|---|---|---|---|
| GLM-4.5-Air 106B (MoE) | 215.9 GB | 144.6 GB | −33.0% | A10 |
| Llama 4 Scout 109B (MoE) | 211.9 GB | 142.2 GB | −32.9% | A10 |
| Qwen3-Next 80B-A3B (MoE) | 161.3 GB | 109.4 GB | −32.2% | A10 |
| Llama 3.3 70B Instruct | 136.9 GB | 91.9 GB | −32.9% | H100 SXM |
| Qwen3 30B-A3B (MoE) | 59.8 GB | 40.2 GB | −32.7% | H100 SXM |
| Gemma 3 27B | 51.5 GB | 34.6 GB | −32.8% | H100 SXM |
| Muse Glimmer 30B | 51.3 GB | 34.5 GB | −32.8% | A10 |
| Qwen3.8 27B | 49.5 GB | 33.3 GB | −32.8% | A10 |
| Gemma 4 26B-A4B (MoE) | 49.0 GB | 32.9 GB | −32.8% | A10 |
| Mistral Small 3.2 24B | 45.3 GB | 30.3 GB | −33.0% | H100 SXM |
| Phi-4 (14B) | 27.3 GB | 18.3 GB | −32.9% | H100 SXM |
| DeepSeek-R1-Distill-Qwen 14B | 26.4 GB | 18.0 GB | −31.9% | H100 SXM |
| Gemma 3 12B | 21.8 GB | 14.6 GB | −32.9% | A10 |
| Llama 3.1 8B Instruct | 14.0 GB | 9.4 GB | −32.8% | H100 SXM |
| Mistral 7B Instruct v0.3 | 14.0 GB | 9.4 GB | −32.7% | H100 SXM |
| Qwen3 8B | 13.9 GB | 9.4 GB | −32.1% | H100 SXM |
| Qwen3 4B 2507 | 7.3 GB | 4.9 GB | −32.2% | A10 |
| SmolLM3 3B | 5.6 GB | 3.8 GB | −32.9% | H100 SXM |
| Llama 3.2 3B Instruct | 5.6 GB | 3.8 GB | −32.8% | A10 |
The 12-bit layout, which decodes with less work and is the one Glyd picks on A10, A100 and H100, takes 23.5 to 24.8% off each (the two layouts).
The same model
The products add their terms in another order than cuBLAS’s, as any two kernels do, so a close call can go either way. Quality was measured where the harness loads the model on one GPU in bf16: perplexity on enwik8 and MMLU on 300 questions, bf16 against Glyd in the same run.
| Model | Perplexity, bf16 | Glyd | MMLU, bf16 | Glyd | Answers as bf16’s | Run |
|---|---|---|---|---|---|---|
| Phi-4 (14B) | 14.7888 | 14.7855 | 76.7% | 76.3% | 99.33% | H100 SXM |
| DeepSeek-R1-Distill-Qwen 14B | 27.1955 | 27.1975 | 78.0% | 78.0% | 100% | H100 SXM |
| Llama 3.1 8B Instruct | 19.5915 | 19.5909 | 71.7% | 72.0% | 99.67% | H100 SXM |
| Mistral 7B Instruct v0.3 | 12.1422 | 12.1473 | 60.7% | 60.7% | 100% | H100 SXM |
| Qwen3 8B | 20.7490 | 20.7428 | 74.0% | 74.3% | 99.67% | H100 SXM |
| SmolLM3 3B | 29.1467 | 29.1422 | 63.3% | 63.3% | 99.67% | H100 SXM |
| Qwen3.8 27B | 15.1946 | 15.1941 | 79.7% | 80.0% | 99.67% | H100 PCIe |
| Gemma 3 12B | — | — | 74.0% | 74.0% | 99.33% | H100 PCIe |
| Qwen3 4B 2507 | 22.4636 | 22.4672 | 71.0% | 71.0% | 99.33% | A10 |
| Llama 3.2 3B Instruct | 25.1298 | 25.1437 | 63.7% | 64.3% | 98.00% | A10 |
Perplexity stays within 0.06% of bf16’s (the widest, Llama 3.2 3B: 25.1298 against 25.1437), and the MMLU answers are bf16’s on 98.0 to 100% of the questions. The differences go both ways. Gemma 3’s perplexity is left out: the harness’s windows start without the BOS token Gemma needs, which puts bf16 and Glyd alike near 12,000. The last four ran with the 12-bit layout the GPU picks (--format auto) and with q, k, v and gate, up merged (--merge); Qwen3.8 27B has a report of its own.
Method
- Sizes:
python sizes.py MODEL_DIR ..., every Linear layer’s matrix in both layouts, unpacked and compared bit for bit. - Quality:
python e2e.py MODEL_DIR --baseline --ppl enwik8 --mmlu 300, bf16 against Glyd in the same run, where the harness loads the model as a causal LM on one GPU. - Models: from Hugging Face; Llama and Gemma from the ungated copies under
unsloth/, the same weights.
The scripts and their options: the GPU docs.
Raw logs
- H100 SXM, Sep 26, 2026: benchmarks/gpu/popular-h100-2026-09-26
- A10, Sep 27, 2026: benchmarks/gpu/open-models-a10-2026-09-27
- H100 PCIe, Sep 27, 2026: benchmarks/gpu/lambda-gpu_1x_h100_pcie-20260926-213544
Every model’s page on this site shows what is measured for it: the models.
Footnotes
-
Only the Linear layers’ matrices are counted, so the bf16 totals sit below the checkpoints’ sizes: GLM-4.5-Air has 215.9 GB of matrices in a 220.94 GB checkpoint. ↩