Glyd
GitHubGet started

Docs

Glyd has two parts. The one this site is about keeps a model’s bf16 weights compressed in GPU memory, about 11 bits a weight instead of 16, and rebuilds the exact values on the GPU: the model is the same to the bit, on less hardware. It installs with one command, curl -LsSf https://getglyd.com/install.sh | sh, and glyd run MODEL starts a chat with a model; from Python, pip install "glyd[gpu]" and glyd.from_pretrained() load a model with its weights packed; engines in C, C++ or Rust call the same kernels through a CUDA library of their own; the benchmarks on this site come from a PyTorch harness in the repository’s gpu/ folder. Underneath it is the codec: a lossless compression library and command line, written in Rust with a C ABI, that installs with Homebrew, cargo or a Python wheel.

The pages

  • Getting started: the one command and glyd run on a Linux machine with an NVIDIA GPU; install glyd[gpu], load a model packed with glyd.from_pretrained(), save it packed, check what fits; reproduce the benchmarks with the harness; install the codec.
  • Serving with vLLM: a local chat with glyd run and glyd serve, vllm serve MODEL --quantization glyd by hand, its options and exact mode, and what it serves against bf16.
  • On the GPU: the two layouts, which GPU runs which, every option of e2e.py, and how to repeat a benchmark from this site.
  • The CUDA library: the GPU kernels for engines in C, C++ and Rust: in the glyd-gpu wheel from v0.27.0, the C header and the license; the downloads and the example of v0.23.0 to v0.26.0.
  • The codec: its levels and modes, the object store, and the Rust, C, Python and Go interfaces.
  • FAQ: quantization, accuracy, GPUs, speed, installing.

The current release

The current release is v0.32.1, of Oct 17, 2026. What it adds:

  • Gemma 4 12B and 31B at :penguin, lossless: glyd run glyd/gemma-4-12B-penguin downloads it (16.1 GB, plus Google’s 0.88 GB draft model); the 31B is 41.3 GB + 0.97 GB. Every weight is Google’s bf16 weight bit for bit: the 12B runs on a 24 GB card, the 31B on a 48 GB card. The changelog
  • v0.32.1: Gemma 4 writes faster. RTX 4080 SUPER, 12B at wren, one user, no draft model: 81.2 to 88.9 tokens a second in chat, 74.7 to 82.1 after a 4K-token prompt; the same output. What changed
  • v0.32.1: glyd fetch MODEL downloads without running; glyd verify downloads first; a glyd/ repo that does not fit the GPU says so and stops. Release notes

Every release and what changed in it: the changelog.

What is not there yet

  • The lossless KV cache for linear attention and a head size of 256, as in Qwen3.5 and Qwen3.8: those keep vLLM’s own cache.
  • Glyd’s own engine on 2026 models: they run on vLLM.
  • Where Glyd is still slower than bf16: the known gaps.
  • The rest of what the package does not do yet: getting started.

Elsewhere on the site