Glyd
GitHubGet started

Report · Sep 26, 2026 · Glyd

Ten popular open models: every matrix 32 to 33% smaller, bit for bit

Every Linear layer's matrix of ten popular open models, from SmolLM3-3B to Llama-3.3-70B-Instruct, packed and unpacked bit for bit: 31.9 to 33.0% smaller.

Glyd’s smallest layout stores the matrices of ten popular open models 31.9 to 33.0% smaller than bf16, and every one of them unpacks to exactly the bf16 values it held. Where a model’s quality could be measured, bf16 and Glyd score the same to noise.

The sizes

gpu/sizes.py packs every Linear layer’s matrix of a model with Glyd’s smallest layout, unpacks it and compares the result with the original bit for bit. These are the totals.1

Model Matrices in bf16, GB Glyd, GB Change, %
Llama-3.3-70B-Instruct 136.9 91.9 −32.9
Qwen3-30B-A3B 59.8 40.2 −32.7
gemma-3-27b-it 51.5 34.6 −32.8
Mistral-Small-3.2-24B-Instruct-2506 45.3 30.3 −33.0
phi-4 27.3 18.3 −32.9
DeepSeek-R1-Distill-Qwen-14B 26.4 18.0 −31.9
Llama-3.1-8B-Instruct 14.0 9.4 −32.8
Mistral-7B-Instruct-v0.3 14.0 9.4 −32.7
Qwen3-8B 13.9 9.4 −32.1
SmolLM3-3B 5.6 3.8 −32.9

The 12-bit layout, the one Glyd picks on A10, A100 and H100, takes 24.7 to 24.8% off each.

The Linear layers' matrices of ten open models in bf16 and with Glyd's smallest layout, in GB. gpu/sizes.py on an H100 SXM, Sep 26, 2026.

The same model

The weights are identical, so the model is too. What can still move is the last bit of a sum: the matrix products round a little differently from bf16’s, as any two kernels do.2 Quality was measured where the harness loads the model on one GPU in bf16, six of the ten: perplexity on enwik8 and MMLU on 300 questions, bf16 against Glyd in the same run.

Model Perplexity, bf16 Perplexity, Glyd MMLU, bf16, % MMLU, Glyd, %
phi-4 14.7888 14.7855 76.7 76.3
DeepSeek-R1-Distill-Qwen-14B 27.1955 27.1975 78.0 78.0
Llama-3.1-8B-Instruct 19.5915 19.5909 71.7 72.0
Mistral-7B-Instruct-v0.3 12.1422 12.1473 60.7 60.7
Qwen3-8B 20.7490 20.7428 74.0 74.3
SmolLM3-3B 29.1467 29.1422 63.3 63.3

Perplexity stays within 0.04% of bf16, and the MMLU answers are bf16’s on 99.3 to 100% of the questions. The differences go both ways.

Method

  • Sizes: every Linear layer’s matrix packed with the smallest layout, unpacked and compared bit for bit (gpu/sizes.py).
  • Quality: perplexity on enwik8 and MMLU on 300 questions, bf16 against Glyd in the same run, where the harness loads the model on one GPU in bf16.
  • Hardware: an H100 SXM, Sep 26, 2026. The driver, CUDA and PyTorch versions are in the logs.

Raw logs

benchmarks/gpu/popular-h100-2026-09-26 in the repository. Every model’s page on this site shows its measured numbers: see the models. Nine more models were measured since: nineteen open models. To measure one yourself: sizes.py in getting started.

Footnotes

  1. Only the Linear layers’ matrices are counted, which is why the bf16 totals sit below the checkpoints’ sizes: Llama-3.3-70B-Instruct has 136.9 GB of matrices in a 141.11 GB checkpoint. ↩

  2. Floating-point addition is not associative, so two kernels that sum the same products in a different order can round the last bit differently. ↩