Tested GPUs
The NVIDIA GPUs each Glyd release is checked and timed on, and Glyd against bf16 on each, release to release.
The GPUs
| GPU | Architecture | Memory | Verified |
|---|---|---|---|
 H100 PCIeHopper · 80 GB | Hopper | 80 GB | Bit for bitlog |
| A100 SXM4 40 GBAmpere · 40 GB | Ampere | 40 GB | Bit for bitlog |
| A10Ampere · 24 GB | Ampere | 24 GB | Bit for bitlog |
| A10GAmpere · 24 GB | Ampere | 24 GB | Bit for bitlog |
| RTX PRO 6000 Blackwell Server EditionBlackwell · 96 GB | Blackwell | 96 GB | Bit for bitlog |
| RTX 4080 SUPERAda · 16 GB | Ada | 16 GB | Bit for bitlog |
Verified: with exact=True, the installed package's output on the GPU is bf16's, bit for bit.
Glyd against bf16, release to release
bf16 and Glyd on the same GPU. A generation step's GPU time after the prompt, and one forward pass over a prompt; lower is better. Change: Glyd against bf16, in percent.
| Weights on the GPU, GB | Generation step, 1 sequence, ms | Prompt of 2,048 tokens, ms | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Release | bf16 | Glyd | Change | bf16 | Glyd | Change | bf16 | Glyd | Change | Log |
| H100 PCIeQwen3-8B, 12-bit layout | ||||||||||
| v0.22.0 | 16.38 | 12.27 | −25 | 12.08 | 11.08 | −8 | 119.1 | 155.2 | +30 | log |
| v0.21.0 | 16.38 | 12.27 | −25 | 12.29 | 11.24 | −9 | — | — | — | log |
| A100 SXM4 40 GBQwen3-8B, 12-bit layout | ||||||||||
| v0.22.0 | 16.38 | 12.27 | −25 | 19.01 | 15.29 | −20 | 176.8 | 197.4 | +12 | log |
| v0.21.0 | 16.38 | 12.27 | −25 | 17.45 | 15.83 | −9 | — | — | — | log |
| A10Qwen3-8B, 12-bit layout | ||||||||||
| v0.22.0 | 16.38 | 12.27 | −25 | 35.70 | 27.75 | −22 | 543.3 | 750.9 | +38 | log |
| A10GQwen3-8B, 12-bit layout | ||||||||||
| v0.22.0 | 16.38 | 12.27 | −25 | 37.13 | 27.80 | −25 | 589.9 | 614.3 | +4 | log |
| RTX PRO 6000 Blackwell Server EditionQwen3-8B, 12-bit layout | ||||||||||
| v0.22.0 | 16.38 | 12.27 | −25 | 13.40 | 11.29 | −16 | 103.0 | 119.6 | +16 | log |
| v0.21.0 | 16.38 | 12.27 | −25 | 13.42 | 11.22 | −16 | 103.2 | 130.5 | +26 | log |
| RTX 4080 SUPERQwen3-4B-Instruct-2507, smallest layout | ||||||||||
| v0.22.0 | 8.04 | 6.00 | −25 | 14.55 | 11.75 | −19 | 197.3 | 198.4 | +1 | log |
| v0.21.0 | 8.04 | 6.00 | −25 | — | — | — | 198.1 | 211.5 | +7 | log |