Questions and answers
Is this quantization?
No. Quantization rounds weights to fewer levels, which changes the model. Glyd stores the same values in fewer bits and gives every one of them back. It is closer to a zip file than to 4-bit. How it works
Does it change the model's answers?
The weights are the model's to the bit. The matrix products round a little differently from bf16's, as any two GPU kernels do, so a close call can go either way: over the models measured, Glyd's MMLU answers are bf16's on 98.0 to 100% of the questions and perplexity stays within 0.06% of bf16's. With glyd.from_pretrained(..., exact=True) the output is bf16's, bit for bit, mixtures of experts included. On Qwen3-4B-Instruct-2507, of 1,000 MMLU answers Glyd's default changed 13, bf16 itself 16 through another attention kernel, and Qwen's FP8 release 46; exact=True none. bf16, FP8 and Glyd compared
Is it smaller than 4-bit quantization?
No. 4-bit formats take 4 bits a weight and their scales; Glyd takes about 11. Glyd is for when you need the model exactly as it was trained, on less hardware. How it compares
Which GPUs?
NVIDIA GPUs from Ampere on, tested on the RTX 4090, RTX 5090, H100 SXM, H100 PCIe, A100, A10, A10G, RTX PRO 6000 and RTX 4080 SUPER. Glyd picks the layout for each GPU. Not measured yet: the B200. Every GPU tested
Which models?
Any model released in bf16, mixtures of experts included: nineteen popular open models are measured, every matrix 32 to 33% smaller; of the 2026 models, measured in vLLM, Qwen3.5-9B's weights are 28.2% smaller on an RTX 4090 and 21.3% on an RTX 5090, and Qwen3.8-27B's 22.8% on an H100 (parts of the newest models are not compressed yet; 33% on every 2026 model is next on the roadmap). FP8 models can shrink 17 to 18% (measured on three), but the GPU kernels for them are not built yet; 4-bit weights have little left to take. Every model
How do I install it?
One command, on Linux with an NVIDIA GPU of the Ampere generation or newer and its driver, 580 or newer: curl -LsSf https://getglyd.com/install.sh | sh, then glyd run Qwen/Qwen3.5-9B downloads the model, packs it as it loads and opens a chat in the terminal and at localhost:8000. On a Mac, or with no NVIDIA GPU, the script installs the compression program and says what glyd run needs. From Python, pip install "glyd[gpu]" (Linux, x86_64 or aarch64; its glyd-gpu wheel carries the GPU kernels, compiled) and glyd.from_pretrained("Qwen/Qwen3.5-9B") load a bf16 model, a mixture of experts too, with its weights packed on the GPU as they arrive. Engines in C, C++ or Rust take the library and its C header from that wheel. The codec alone installs with pip install glyd, Homebrew or cargo. The current release is v0.32.1, of Oct 17, 2026. Getting started
Is it faster than bf16?
What Glyd claims is memory: on the 2026 models measured in vLLM, the weights are 21.3 to 28.2% smaller than bf16's. Glyd will claim speed only where it is faster than vLLM at every load, with less memory; at full load on an H100, each Qwen3.8-27B token took 1.43 times bf16's time. The earlier releases' speed runs on 2025 models, through generate() and in vLLM, stay on the benchmarks page with their logs. The v0.29.0 runs below measured weights 21.3 to 28.2% smaller than bf16's; v0.29.1 compresses additional loaded weights. One-third savings across every model and GPU remain a target. The v0.28.0 H100 run measured each token at full load taking 1.43 times bf16's time. Earlier measurements remain listed with their release. The benchmarks
Does it work with vLLM?
Yes, from v0.26.0: glyd run MODEL starts vLLM with settings worked out from your GPU and opens a chat, and for a setup of your own, pip install "glyd[vllm]", then vllm serve MODEL --quantization glyd. vLLM keeps the weights packed, the embedding and the output layer too from v0.28.0, and gives the memory they save to its KV cache: 1.04 to 2.11 times bf16's on the GPUs measured, and 1.28 to 1.33 times the requests a second at saturation on an L4, an A10 and an A100 with Qwen3-14B (0.98 times with Qwen3-8B on an A100 and 0.89 to 0.93 times on a GH200, with v0.27.0; Qwen3-32B on an 80 GB H100 SXM served 1.56 times with every layer packed, and from v0.28.0 a mixture of experts on an H100 SXM 1.07 times). With exact the logits are bf16's, bit for bit. On 2026 models: Qwen3.8-27B on an H100 held 1.67 times bf16's KV cache tokens. Glyd's own engine ships with v0.29.0, off unless you ask for it; it does not run 2026 models yet, so they run on vLLM. An engine of your own can call the kernels through the CUDA library, from C or its Rust crate. Serving with vLLM
Does it compress the KV cache too?
Yes, in vLLM from v0.27.0: the KV cache is held in fewer bits and every value comes back bit for bit, 1.25 to 1.30 times vLLM's tokens in the same memory on five models, and decoding is faster than with vLLM's own cache at every batch measured on an A100, an L4, an H100 and a GH200 (with Glyd's weights as well, an L4 served 1.59 times bf16's requests a second with 2.64 times the KV tokens; with long prompts on an H100 each generated token takes 1.12 times as long). glyd serve and vllm serve --quantization glyd use it by default on those four GPUs; glyd run keeps vLLM's own cache unless GLYD_KV is set. It does not hold Qwen3.5's and Qwen3.8's attention yet (linear attention and a head size of 256): those models keep vLLM's own cache. In the PyTorch harness, Qwen2.5-7B-Instruct's cache for 16,384 tokens takes 651 MB instead of 947, and comes back bit for bit. The lossless KV cache
Is it open source?
No, it is source-available. From v0.29.0 all of Glyd is under the Business Source License 1.1: free forever for personal and non-commercial use (including education, research and nonprofits) on computers you own or rent for yourself, and for anyone to try, test and develop with. Commercial use needs a license: suryakoritala@getglyd.com. Each version becomes Apache-2.0 four years after its release. Releases up to v0.28 keep the licenses they shipped with (the codec was BSD-3-Clause OR GPL-2.0). The license