Glyd
GitHubGet started

Report · · Glyd

Ten popular open models: every matrix 32 to 33% smaller, bit for bit

Every Linear layer's matrix of ten popular open models, from SmolLM3 3B to Llama 3.3 70B, packed and unpacked bit for bit: 31.9 to 33.0% smaller.

Glyd’s tiered layout stores the matrices of ten popular open models 31.9 to 33.0% smaller than bf16, and every one of them unpacks to exactly the bf16 values it held. Where a model’s quality could be measured, bf16 and Glyd score the same to noise.

The sizes

gpu/sizes.py packs every Linear layer’s matrix of a model with Glyd’s tiered layout, unpacks it and compares the result with the original bit for bit. These are the totals.1

Model Matrices in bf16 Glyd Change
Llama 3.3 70B Instruct 136.9 GB 91.9 GB −32.9%
Qwen3 30B-A3B (MoE) 59.8 GB 40.2 GB −32.7%
Gemma 3 27B 51.5 GB 34.6 GB −32.8%
Mistral Small 3.2 24B 45.3 GB 30.3 GB −33.0%
Phi-4 (14B) 27.3 GB 18.3 GB −32.9%
DeepSeek-R1-Distill-Qwen 14B 26.4 GB 18.0 GB −31.9%
Llama 3.1 8B Instruct 14.0 GB 9.4 GB −32.8%
Mistral 7B Instruct v0.3 14.0 GB 9.4 GB −32.7%
Qwen3 8B 13.9 GB 9.4 GB −32.1%
SmolLM3 3B 5.6 GB 3.8 GB −32.9%

The 12-bit layout, which decodes with less work and is the one Glyd picks on A10, A100 and H100, takes 24.7 to 24.8% off each.

The Linear layers' matrices of ten open models in bf16 and with Glyd's tiered layout, in GB. gpu/sizes.py on an H100 SXM, Sep 26, 2026.

The same model

The weights are identical, so the model is too. What can still move is the last bit of a sum: the matrix products add their terms in another order than cuBLAS’s, as any two kernels do.2 Quality was measured where the harness loads the model on one GPU in bf16, six of the ten: perplexity on enwik8 and MMLU on 300 questions, bf16 against Glyd in the same run.

Model Perplexity, bf16 Glyd MMLU, bf16 Glyd
Phi-4 (14B) 14.7888 14.7855 76.7% 76.3%
DeepSeek-R1-Distill-Qwen 14B 27.1955 27.1975 78.0% 78.0%
Llama 3.1 8B Instruct 19.5915 19.5909 71.7% 72.0%
Mistral 7B Instruct v0.3 12.1422 12.1473 60.7% 60.7%
Qwen3 8B 20.7490 20.7428 74.0% 74.3%
SmolLM3 3B 29.1467 29.1422 63.3% 63.3%

Perplexity stays within 0.04% of bf16, and the MMLU answers are bf16’s on 99.3 to 100% of the questions. The differences go both ways.

Method

  • Sizes: every Linear layer’s matrix packed with the tiered layout, unpacked and compared bit for bit (gpu/sizes.py).
  • Quality: perplexity on enwik8 and MMLU on 300 questions, bf16 against Glyd in the same run, where the harness loads the model on one GPU in bf16.
  • Hardware: an H100 SXM, Sep 26, 2026. The driver, CUDA and PyTorch versions are in the logs.

Raw logs

benchmarks/gpu/popular-h100-2026-09-26 in the repository. Every model’s page on this site shows its measured numbers: see the models.

Footnotes

  1. Only the Linear layers’ matrices are counted, which is why the bf16 totals sit below the checkpoints’ sizes: Llama 3.3 70B has 136.9 GB of matrices in a 141.11 GB checkpoint. ↩

  2. Floating-point addition is not associative, so two kernels that sum the same products in a different order can round the last bit differently. ↩