Report · · Glyd
Against DFloat11 and ZipServ on an RTX 4080 SUPER
Qwen3-8B on a 16 GB RTX 4080 SUPER: Glyd generates 2.5 to 3.4 times DFloat11's tokens a second at the same size, and its kernels keep pace with ZipServ's.
Glyd is not the only lossless format for model weights. DFloat11 and ZipServ also keep every weight bit for bit at about 11 bits instead of 16. What separates the three is where the weights are decoded, and that decides the speed. This report runs them on the same 16 GB RTX 4080 SUPER.
Three ways to decode
| Method | Bits a weight | Same model? | Where the weights are decoded |
|---|---|---|---|
| Glyd (open source, 2026) | 10.8 or 12.0 | Yes, bit for bit | Inside the matrix multiply, in registers |
| DFloat11 (NeurIPS 2025) | about 11 | Yes, bit for bit | Each block decompressed to bf16 in memory before it runs |
| ZipServ (ASPLOS 2026) | 11.35 | Yes, bit for bit | Inside the matrix multiply |
| 4-bit quantization (GGUF Q4, AWQ, GPTQ) | about 4.5 | No, rounded | Smallest by far, and a different model: its answers change |
End to end against DFloat11
Qwen3-8B takes 16.38 GB in bf16, which leaves no room to run on a 16 GB card. Both DFloat11 and Glyd hold its weights in about 11.2 GB. DFloat11 decompresses each block to bf16 in memory before running it; Glyd decodes inside the product, so the bf16 weights never exist in memory.
| Sequences a step | DFloat11, tokens/s | Glyd, tokens/s | Glyd against DFloat11 |
|---|---|---|---|
| 1 | 13.8 | 47.2 | 3.4× |
| 8 | 104.7 | 360.2 | 3.4× |
| 32 | 387.5 | 1268.6 | 3.3× |
| 64 | 746.2 | 1897.2 | 2.5× |
| Sequences a step | DFloat11 | Glyd |
|---|---|---|
| 1 | 13.8 | 47.2 |
| 8 | 104.7 | 360.2 |
| 32 | 387.5 | 1,268.6 |
| 64 | 746.2 | 1,897.2 |
The matrix multiply against ZipServ and cuBLAS
ZipServ decodes inside the matrix multiply too, so the comparison that matters is kernel against kernel. These are Qwen3-8B’s layer 18 matrices, timed the way ZipServ times itself: microseconds a call with the L2 cache flushed. Each cell is cuBLAS on bf16 / ZipServ / Glyd; lower is better.
| Matrix | Bits a weight, ZipServ / Glyd | 1 token | 16 tokens | 32 tokens | 64 tokens |
|---|---|---|---|---|---|
| q_proj, o_proj (4096 × 4096) | 11.35 / 10.81 | 73 / 57 / 56 | 73 / 57 / 58 | 74 / 58 / 62 | 77 / 69 / 73 |
| k_proj (1024 × 4096) | 11.43 / 10.83 | 23 / 22 / 19 | 27 / 23 / 21 | 26 / 23 / 24 | 28 / 25 / 30 |
| gate_proj (12288 × 4096) | 11.35 / 10.76 | 175 / 151 / 144 | 204 / 152 / 148 | 235 / 155 / 153 | 225 / 162 / 172 |
| down_proj (4096 × 12288) | 11.35 / 10.75 | 176 / 153 / 147 | 208 / 154 / 150 | 228 / 155 / 153 | 214 / 187 / 181 |
Glyd is the fastest of the three in 10 of the 16 cells, and ZipServ in the other 6; both beat cuBLAS on bf16 almost everywhere. Glyd’s matrices are also the smaller, at 10.75 to 10.83 bits a weight against ZipServ’s 11.35 to 11.43. Glyd ran its tiered layout at 1 to 32 tokens and its 12-bit layout at 64.
Against bf16 on the same card
For scale, Glyd against bf16 itself on this GPU: Qwen2.5-7B, GPU time of a generated token with q, k, v and gate, up merged as vLLM runs them. Glyd takes 9 to 28% less time from 1 to 64 sequences a step.
Method
- End to end: tokens per second generating with Qwen3-8B, DFloat11 and Glyd on the same RTX 4080 SUPER.
- Kernels: microseconds a call of each matrix product with the L2 cache flushed, as ZipServ times itself.
- Against bf16: GPU time of a generated token apart from the prompt, 17 steps less 1, over 16; q, k, v as one product and gate, up as another, for bf16 and Glyd alike.
- Exactness: every matrix packed, unpacked and compared bit for bit.
The logs of every run are in the repository, under benchmarks/gpu.