Lossless AI compression: 33% less GPU memory, bit for bit.
Glyd stores the weights of open models like Qwen3.5, Gemma 4, Nemotron 3 and GLM-5 in about 11 bits instead of 16 and rebuilds the exact values on the GPU. Every weight comes back exactly, so you run the same model on less hardware.
Glyd stores the weights of open models in about 11 bits instead of 16 and rebuilds the exact values on the GPU. The same model on less hardware.
The black line is the GPU's 24 GB. vLLM 0.30.0, Oct 5, 2026.
Measured on open models from
Qwen
Meta
Google
Mistral AI
DeepSeek
Hugging Face
Microsoft
Z.ai
Will it fit on your GPU?
Pick your GPU's memory. Weights plus an 8K-token KV cache and 1.5 GB for the runtime, bf16 against Glyd.
Weights, an 8K-token KV cache and the runtime, bf16 against Glyd.
| Model | bf16, GB | Glyd, GB | Memory it needs, against this GPU | On one 24 GB GPU |
|---|---|---|---|---|
| gemma-4-12B-itest. | 23.9 | 16.1 | Fits only with Glyd | |
| gemma-3-12b-itest. | 24.4 | 16.4 | Fits only with Glyd | |
| phi-4est. | 29.3 | 19.7 | Fits only with Glyd | |
| DeepSeek-R1-Distill-Qwen-14Best. | 29.5 | 20.1 | Fits only with Glyd | |
| Qwen3.5-9Best. | 19.3 | 13.9 | Fits either way | |
 MiMo-V2.6-Distill-Qwen-9Best. | 18.8 | 12.7 | Fits either way | |
| Qwen3-8Best. | 16.4 | 11.2 | Fits either way | |
| Mistral-Small-3.2-24B-Instruct-2506est. | 48.0 | 32.2 | Too big for one |
Measured on 2026 models
The weights as vLLM gives them, bf16 and Glyd on the same GPU. On the 24 GB RTX 4090, bf16 did not start with vLLM's default settings; Glyd runs Qwen3.5-9B there with a 189,440-token context.
The memory it frees holds more requests: on two H100s, Qwen2.5-72B has room for about 27 conversations of 4,096 tokens at once with Glyd, and about 2 in bf16 (the report). Parts of the newest models are not compressed yet: 33% on every 2026 bf16 model is the work on the roadmap.
All benchmarks| Weights, GiB | |||
|---|---|---|---|
| GPU | bf16 | Glyd | Change, % |
| RTX 4090Qwen3.5-9B log | 17.66 | 12.68 | −28 |
| RTX 5090Qwen3.5-9B log | 17.66 | 13.89 | −21 |
| H100 SXMQwen3.8-27B log | 50.22 | 38.77 | −23 |
Lower is smaller. vLLM 0.30.0, Glyd against vLLM's own bf16; Qwen3.8-27B text only. The measurements.
Earlier: tokens a second through generate()
v0.25.1, Sep 30, 2026: Qwen3-8B and Qwen3-32B, Glyd's default against bf16 compiled the same way. Generating a token reads every weight once, so reading a third fewer bytes saves time; at 8 sequences on an A10, and to a prompt's first token, Glyd took longer than bf16 (the known gaps).
How fast it responds| Tokens a second, Glyd against bf16, % | |||
|---|---|---|---|
| GPU | 1 sequence | 8 | 32 |
| L4Qwen3-8B | +27 | +27 | +22 |
| A10Qwen3-8B | +31 | −6 | +11 |
| A100 SXM4 40 GBQwen3-8B, v0.25.1 | +23 | +5 | +4 |
| A100 SXM4 40 GBQwen3-14B, v0.25.1 | +27 | +5 | +5 |
| GH200Qwen3-8B, v0.25.1 | +6 | +21 | +20 |
| GH200Qwen3-32B, v0.25.1 | +19 | +33 | +30 |
Higher is faster. Through generate(), Glyd's default against bf16 compiled the same way: a 128-token prompt, 256 new tokens. How fast it responds.
How it works, in three stepsHow it works
The full explanation- 01
Fewer bits
Glyd stores each bf16 weight in about 11 bits instead of 16, so a model’s weights take about a third less GPU memory.About 11 bits a weight instead of 16: a third less GPU memory.
- 02
Every value exact
Nothing is rounded. Every weight comes back as the same 16 bits, so it is the same model, not an approximation of it.Nothing is rounded: every weight comes back as the same 16 bits.
- 03
The GPU works from the compact form
The GPU reads the compact weights and rebuilds the exact values as it works, so the model never takes its bf16 size in GPU memory.The GPU reads the compact weights and rebuilds the exact values as it works.
How it compares
Lossless weight compression is an active research area. Here is where Glyd sits, on numbers we measured ourselves where we could.
| Method | Bits a weight | Same model? |
|---|---|---|
| Glyd2026. Source available, BUSL-1.1: the license | 10.9 / 12.0 | Yes, bit for bit |
| DFloat11NeurIPS 2025 | 10.9 | Yes, bit for bit |
| ZipServASPLOS 2026 | 11.4 | Yes, bit for bit |
| 4-bit quantizationGGUF Q4, AWQ, GPTQ | 4 + scales | No, rounded: smallest by far, and its answers change |
Bits a weight measured on Qwen3-8B: the whole model for Glyd's two layouts and DFloat11, the matrices for ZipServ. On the same RTX 4080 SUPER Glyd made 3.4× DFloat11's tokens a second at one sequence, and in 10 of 16 products its kernels were the fastest of Glyd, ZipServ and cuBLAS (log).
Latest
All reports- Release · Oct 17, 2026Glyd v0.32.1: Gemma 4 writes faster, glyd fetch, clearer refusalsGlyd v0.32.1: Gemma 4 12B at wren from 81.2 to 88.9 tokens a second on an RTX 4080 SUPER, glyd fetch, and a clear stop where a glyd/ repo does not fit.
- Release · Oct 17, 2026Glyd v0.32.0: Gemma 4 12B and 31B, losslessGlyd v0.32.0: Gemma 4 12B and 31B at :penguin, every weight Google's bf16 weight bit for bit, the 12B on a 24 GB card and the 31B on a 48 GB card.
- Release · Oct 16, 2026Glyd v0.31.0: Gemma 4 31B and 12B, exact at 4.5 bits, with draftingGlyd v0.31.0: Gemma 4 31B and 12B at :wren, Google's QAT weights bit for bit at 4.5 bits per weight, with Google's draft models.
Questions
All questionsIs this quantization?
No. Quantization rounds weights to fewer levels, which changes the model. Glyd stores the same values in fewer bits and gives every one of them back. It is closer to a zip file than to 4-bit.
Does it change the model's answers?
The weights are the model's to the bit. The matrix products round a little differently from bf16's, as any two GPU kernels do, so a close call can go either way: over the models measured, Glyd's MMLU answers are bf16's on 98.0 to 100% of the questions and perplexity stays within 0.06% of bf16's. With glyd.from_pretrained(..., exact=True) the output is bf16's, bit for bit, mixtures of experts included. On Qwen3-4B-Instruct-2507, of 1,000 MMLU answers Glyd's default changed 13, bf16 itself 16 through another attention kernel, and Qwen's FP8 release 46; exact=True none.
Which GPUs?
NVIDIA GPUs from Ampere on, tested on the RTX 4090, RTX 5090, H100 SXM, H100 PCIe, A100, A10, A10G, RTX PRO 6000 and RTX 4080 SUPER. Glyd picks the layout for each GPU. Not measured yet: the B200.
How do I install it?
One command, on Linux with an NVIDIA GPU of the Ampere generation or newer and its driver, 580 or newer: curl -LsSf https://getglyd.com/install.sh | sh, then glyd run Qwen/Qwen3.5-9B downloads the model, packs it as it loads and opens a chat in the terminal and at localhost:8000. On a Mac, or with no NVIDIA GPU, the script installs the compression program and says what glyd run needs. From Python, pip install "glyd[gpu]" (Linux, x86_64 or aarch64; its glyd-gpu wheel carries the GPU kernels, compiled) and glyd.from_pretrained("Qwen/Qwen3.5-9B") load a bf16 model, a mixture of experts too, with its weights packed on the GPU as they arrive. Engines in C, C++ or Rust take the library and its C header from that wheel. The codec alone installs with pip install glyd, Homebrew or cargo. The current release is v0.32.1, of Oct 17, 2026.
Try it on your model
One command installs Glyd with vLLM, and glyd run downloads a model, packs its weights as they load and opens a chat in the terminal and at localhost:8000. It needs Linux, an NVIDIA GPU of the Ampere generation or newer, and its driver, 580 or newer. On a Mac, or with no NVIDIA GPU, the script installs the compression program and says what glyd run needs (getting started).
In Python, pip install "glyd[gpu]" loads any model released in bf16 with its weights packed on the GPU, every one bit for bit. Engines in C, C++ or Rust call the same kernels through the CUDA library.
One command, on Linux with an NVIDIA GPU
curl -LsSf https://getglyd.com/install.sh | sh
glyd run Qwen/Qwen3.5-9BOr in Python
pip install "glyd[gpu]"
import glyd
model = glyd.from_pretrained("Qwen/Qwen3.5-9B")