Related work
Coding a bf16 weight's exponent losslessly is not new. What Glyd adds is the combination: the smallest code decoded inside the product, the KV cache too, and the same result every run.
The work Glyd builds on
- DFloat11 (Zhang et al., NeurIPS 2025): Huffman codes for the exponent, about 30% smaller, outputs bit for bit. Each transformer block is decompressed to bf16 before it runs, which its authors report as about twice bf16’s time at one sequence.
- ZipServ (ASPLOS 2026): A fixed-length bitmap code decoded straight into the tensor cores’ registers inside the product: up to 30% smaller, up to 2.21 times cuBLAS’s speed per product and 1.22 times vLLM’s end to end on GDDR GPUs.
- Approaching Shannon Bound with Lossless LLM Weight Compression (Tan et al., ISCA 2026): ANS codes within 0.01 to 0.05 bits of the entropy bound for bf16, FP8 and integer formats, decoded a tile at a time in SGLang.
- SplitZip (2026): The KV cache’s exponents as 4-bit codes into the 16 commonest, with escapes, for moving the cache between servers: the scheme of Glyd’s 12-bit layout.
- ZipNN (2024): Lossless compression of model weights for storage.
- NeuZip (2024): Compressed weights to save memory in training.
- Huff-LLM (2025): Huffman-coded weights decoded in hardware.
- Lossless compression of weights, checkpoints and K/V caches in low-precision formats (2025): The same coding applied to weights, checkpoints and the KV cache in low-precision formats.
- Unweight (Cloudflare, 2026): Lossless compression of MLP weights.
- dietgpu (Jeff Johnson, Meta, 2021): An ANS entropy coder on the GPU and a lossless codec for float16 and bfloat16 that codes the exponent, used to compress what GPUs send each other in training.
- fpzip (Lindstrom and Isenburg, LLNL, 2006): Lossless and lossy compression of floating-point arrays by prediction and entropy coding.
- FPC, MPC and others (Martin Burtscher’s lab, Texas State): Fast lossless compressors for floating-point data on CPUs and GPUs, from 2009 on.
Where Glyd differs
- The smallest code decoded in the product. 10.80 bits a weight (32.5% off bf16) decoded inside the matrix multiply itself; the others decode about 30% off that way, or reach the bound with an ANS decoder.
- The KV cache too. The cache is held compressed and attention reads its pages: 31% smaller.
- The same result every run. Products split across blocks are summed in a fixed order.
- Beyond the GPU. The same coding makes the model smaller on disk, a fine-tune against its base smaller still, and training checkpoints with their optimizer state.
How the exponent's 8 bits come down to about 2.6: why a bf16 exponent carries only 2.6 bits. How the two layouts decode: how it works and on the GPU.
Side by side
On one 16 GB RTX 4080 SUPER with Qwen3-8B, whose 16.38 GB of bf16 does not fit it, Glyd holds the weights at DFloat11’s size and generates 2.9 to 3.4 times its tokens a second. Against ZipServ, product against product on the same matrices, Glyd takes 5% fewer bytes, is as fast or faster at 1 to 16 tokens, and is behind on most matrices at 32 and 64, by up to 20%. The numbers, the method and the raw logs: the head-to-head report and the benchmarks.