Same weights. Fewer bits. Decoded right where they are used.
Glyd keeps a model's bf16 weights in GPU memory at about 11 bits each instead of 16, and turns them back into the exact same values inside the matrix multiply. The model is unchanged. It just takes a third less memory.
A bf16 weight is 16 bits
One sign bit, eight exponent bits and seven mantissa bits. In a trained model the sign and mantissa look random: there is nothing to squeeze out of them.
The exponent is different. Most weights have similar sizes, so a handful of exponent values cover almost all of them. Its eight bits carry only about 2.6 bits of information.
Glyd rewrites only the exponent
Common exponents get short codes. The rare ones are stored exactly in a small side list. Sign and mantissa stay as they are. Nothing is rounded, so decoding gives back the identical 16 bits.
Two layouts, picked per GPU: the tiered one is the smallest; the 12-bit one decodes with the least work, for GPUs whose memory is fast enough that decoding becomes the bottleneck.
Close to the limit: no code that looks at each tensor on its own gets below about 10.6 bits a weight.
Decoded inside the matrix multiply
Each generated token reads every weight once, so the speed is set by how many bytes cross from memory. Glyd reads the compressed bytes and decodes them in the GPU's registers, straight into the tensor cores. The bf16 weights never exist in memory again.
A third fewer bytes cross, which is why Glyd is often faster than bf16, not slower.
- GPU memorycompressed
- Decompressa kernel
- GPU memorybf16 copy of the block
- Multiplyreads bf16 again
- GPU memorycompressed
- Registersdecode into the tensor cores
- Resultno bf16 copy in memory
Is it the same model?
The weights are identical: every matrix of ten popular models packed, unpacked and compared bit for bit.
The results match to noise. The products add up in another order than cuBLAS's, as between any two kernels, so scores move a hair in either direction: perplexity within 0.04%, MMLU answers the same on 99.3 to 100% of questions.
| Model | Perplexity, bf16 → Glyd | MMLU, bf16 → Glyd |
|---|---|---|
| Phi-4 (14B) | 14.7888 → 14.7855 | 76.7% → 76.3% |
| DeepSeek-R1-Distill-Qwen 14B | 27.1955 → 27.1975 | 78.0% → 78.0% |
| Llama 3.1 8B Instruct | 19.5915 → 19.5909 | 71.7% → 72.0% |
| Mistral 7B Instruct v0.3 | 12.1422 → 12.1473 | 60.7% → 60.7% |
| Qwen3 8B | 20.7490 → 20.7428 | 74.0% → 74.3% |
| SmolLM3 3B | 29.1467 → 29.1422 | 63.3% → 63.3% |
| Qwen3.8 27B | 15.1946 → 15.1941 | 79.7% → 80.0% |
| Gemma 3 12B | — | 74.0% → 74.0% |
| Qwen3 4B 2507 | 22.4636 → 22.4672 | 71.0% → 71.0% |
| Llama 3.2 3B Instruct | 25.1298 → 25.1437 | 63.7% → 64.3% |
Perplexity on enwik8, MMLU on 300 questions; H100 SXM, H100 PCIe and A10, Sep 27, 2026. Gemma 3's perplexity is left out: the windows start without the token Gemma needs.
Questions people ask
Is this quantization?
No. Quantization rounds weights to fewer levels, which changes the model. Glyd stores the same values in fewer bits and gives every one of them back. It is closer to a zip file than to 4-bit.
Is it smaller than 4-bit quantization?
No. 4-bit formats take about 4.5 bits a weight; Glyd takes about 11. Glyd is for when you need the model exactly as it was trained, on less hardware.
Which models and GPUs?
Any model released in bf16. Measured on RTX 4080 SUPER, RTX A6000, A10, A100 and H100. FP8 models can shrink 16 to 18% (measured on three); GPU kernels for them are not built yet.
Is it open source?
The codec is BSD-3-Clause OR GPL-2.0. The GPU kernels and the store are source-available under the Business Source License 1.1.