Why a bf16 exponent carries only 2.6 bits

Most open models you can run on one GPU are released in bf16: 16 bits a weight. Glyd stores the same weights in about 11 bits, and gives every one of them back exactly. There is no trick in that, only a gap between the bits a number is stored in and the information it carries. In a bf16 weight the gap is almost all in one place: the exponent.

Sixteen bits, three fields

A bf16 number is a sign bit, eight exponent bits and seven mantissa bits. The exponent says how big the number is, as a power of two; the mantissa says where it sits between two powers of two; the sign says which side of zero.

SEEEEEEEEMMMMMMM

Bits of storage and bits of information

A field’s width is how many bits it is stored in. The information it carries is how many bits it takes to say which value it holds, on average, when you know how often each value turns up: its entropy.¹ A field whose 256 values all turn up equally often carries its full 8 bits. A field that almost always holds one of a handful of values carries far fewer, because common values can get short codes and rare ones long codes, and the average comes out small.

That is the difference that matters here. Eight bits can name 256 exponents; a trained model does not use them evenly.

Why the exponent is predictable

In a trained model most weights have similar sizes, so a handful of exponent values cover almost all of them. The exponent’s eight bits end up carrying only about 2.6 bits of information.

The sign and the mantissa are another matter. In a trained model they look random: a weight is as likely to be negative as positive, and its seven mantissa bits are as good as noise. There is nothing to squeeze out of them, and no lossless code gets them below the 8 bits they take.

What that adds up to

One sign bit, seven mantissa bits and about 2.6 bits of exponent make about 10.6 bits a weight. That is the floor: no code that looks at each tensor on its own gets below about 10.6 bits a weight, so no lossless code takes more than about 34% off bf16 weights. The same holds for the KV cache, whose keys and values are bf16 numbers too.

Glyd gets close. Its tiered layout takes 10.72 to 10.89 bits a weight, 32 to 33% smaller than bf16; across ten popular open models every matrix came out 31.9 to 33.0% smaller (the report).

How Glyd codes the exponent

Glyd rewrites only the exponent. Common exponents get short codes; the rare ones are stored exactly in a small side list. The sign and mantissa stay as they are, so decoding gives back the identical 16 bits. There are two layouts, and Glyd picks one per GPU:

bf1616 bits

exponent, 8 bitssign + mantissa, 8 bits, as they were

As the model ships.

Glyd, 12-bit layout12.04 bits · 25% smaller

4-bit codesign + mantissa, 8 bits, as they were

The 15 commonest exponents as 4-bit codes, the rest in a side list. Used on A10, A100 and H100.

Glyd, tiered layout10.72 to 10.89 bits · 32 to 33% smaller

2.8 bitssign + mantissa, 8 bits, as they were

Codes in 2-bit steps: the commonest exponents take 2 bits. Used on RTX 40-series (Ada).

Close to the limit: no code that looks at each tensor on its own gets below about 10.6 bits a weight.

The tiered layout is the smallest. The 12-bit layout decodes with the least work, for GPUs whose memory is fast enough that decoding becomes the bottleneck. Either way the weights are decoded in the GPU’s registers inside the matrix multiply, so no bf16 copy is ever made (how it works).

Measure it yourself

The exponent’s entropy is easy to measure on any bf16 tensor. This counts how often each of the 256 exponent values turns up and computes the entropy of that distribution:

import torch

def exponent_bits(w: torch.Tensor) -> float:
    """Information in the 8 exponent bits of a bf16 tensor, in bits a weight."""
    e = (w.contiguous().view(torch.int16) >> 7) & 0xFF   # bits 14 to 7
    counts = torch.bincount(e.flatten().long(), minlength=256).double()
    p = counts[counts > 0] / counts.sum()
    return float(-(p * p.log2()).sum())

Run it on the weight matrices of a model released in bf16, for example one loaded with safetensors.torch.load_file, and compare the result with the 8 bits the exponent takes.

Where it does not apply

The saving comes from bf16’s generous exponent. Models released in FP8 have less to take, 16 to 18% as measured on three of them, and Glyd’s GPU kernels for FP8 are not built yet. Models released in 4-bit, such as gpt-oss, DeepSeek-V4 and Kimi K3, have their weights already rounded to 4 bits, with little left to take. 4-bit quantization is smaller than Glyd, about 4.5 bits a weight, but it rounds the weights and changes the model’s answers; Glyd is for when you need the model exactly as it was trained, on less hardware.

Footnotes

1. Shannon entropy: for values that turn up with probabilities p, the sum of −p × log₂ p over the values. No lossless code can average fewer bits a value than that. ↩