Glyd
GitHubGet started

Report · Sep 27, 2026 · Glyd

Nineteen open models: every matrix 32 to 33% smaller, bit for bit

Every Linear layer's matrix of nineteen open models, SmolLM3-3B to GLM-4.5-Air, packed and unpacked bit for bit: 31.9 to 33.0% smaller, quality on ten.

Ten popular open models were measured first; nine more have been since, from GLM-4.5-Air and Llama-4-Scout-17B-16E-Instruct down to Llama-3.2-3B-Instruct. On all nineteen, Glyd’s smallest layout stores every Linear layer’s matrix 31.9 to 33.0% smaller than bf16, and every matrix unpacks to exactly the bf16 values it held. Where quality was measured, on ten of them, bf16 and Glyd score the same to noise.

The sizes

sizes.py packs every Linear layer’s matrix of a model, unpacks it and compares the result with the original bit for bit. Two things it counts that a quick sum would miss: a layer’s experts kept as one tensor (Gemma 4, Llama 4) count as a matrix an expert, and every projection is counted, the linear-attention inputs of Qwen3-Next and Qwen3.8 included.1

ModelMatrices in bf16, GBGlyd, GBChange, %Run
GLM-4.5-Air215.9144.6−33.0A10
Llama-4-Scout-17B-16E-Instruct211.9142.2−32.9A10
Qwen3-Next-80B-A3B-Instruct161.3109.4−32.2A10
Llama-3.3-70B-Instruct136.991.9−32.9H100 SXM
Qwen3-30B-A3B59.840.2−32.7H100 SXM
gemma-3-27b-it51.534.6−32.8H100 SXM
Muse-Glimmer-30B51.334.5−32.8A10
Qwen3.8-27B49.533.3−32.8A10
gemma-4-26B-A4B-it49.032.9−32.8A10
Mistral-Small-3.2-24B-Instruct-250645.330.3−33.0H100 SXM
phi-427.318.3−32.9H100 SXM
DeepSeek-R1-Distill-Qwen-14B26.418.0−31.9H100 SXM
gemma-3-12b-it21.814.6−32.9A10
Llama-3.1-8B-Instruct14.09.4−32.8H100 SXM
Mistral-7B-Instruct-v0.314.09.4−32.7H100 SXM
Qwen3-8B13.99.4−32.1H100 SXM
Qwen3-4B-Instruct-25077.34.9−32.2A10
SmolLM3-3B5.63.8−32.9H100 SXM
Llama-3.2-3B-Instruct5.63.8−32.8A10

The 12-bit layout, the one Glyd picks on A10, A100 and H100, takes 23.5 to 24.8% off each (the two layouts).

The Linear layers' matrices of nineteen open models in bf16 and with Glyd's smallest layout, in GB. gpu/sizes.py on an H100 SXM (Sep 26, 2026) and an A10 (Sep 27, 2026).

The same model

The products round a little differently from bf16’s, as any two kernels do, so a close call can go either way. Quality was measured where the harness loads the model on one GPU in bf16: perplexity on enwik8 and MMLU on 300 questions, bf16 against Glyd in the same run.

ModelPerplexity, bf16Perplexity, GlydMMLU, bf16, %MMLU, Glyd, %Answers as bf16’s, %Run
phi-414.788814.785576.776.399.33H100 SXM
DeepSeek-R1-Distill-Qwen-14B27.195527.197578.078.0100.00H100 SXM
Llama-3.1-8B-Instruct19.591519.590971.772.099.67H100 SXM
Mistral-7B-Instruct-v0.312.142212.147360.760.7100.00H100 SXM
Qwen3-8B20.749020.742874.074.399.67H100 SXM
SmolLM3-3B29.146729.142263.363.399.67H100 SXM
Qwen3.8-27B15.194615.194179.780.099.67H100 PCIe
gemma-3-12b-it——74.074.099.33H100 PCIe
Qwen3-4B-Instruct-250722.463622.467271.071.099.33A10
Llama-3.2-3B-Instruct25.129825.143763.764.398.00A10

Perplexity stays within 0.06% of bf16’s, and the MMLU answers are bf16’s on 98.0 to 100% of the questions; the differences go both ways. Qwen3.8-27B has a report of its own.

Method

  • Sizes: python sizes.py MODEL_DIR ..., every Linear layer’s matrix in both layouts, unpacked and compared bit for bit.
  • Quality: python e2e.py MODEL_DIR --baseline --ppl enwik8 --mmlu 300, bf16 against Glyd in the same run, where the harness loads the model as a causal LM on one GPU.
  • Models: from Hugging Face; Llama and Gemma from the ungated copies under unsloth/, the same weights.

The scripts and their options: the GPU docs.

Raw logs

Every model’s page on this site shows what is measured for it: the models.

Footnotes

  1. Only the Linear layers’ matrices are counted, so the bf16 totals sit below the checkpoints’ sizes: GLM-4.5-Air has 215.9 GB of matrices in a 220.94 GB checkpoint. ↩