Blog
Release posts, measured results and plans for making AI model weights smaller without changing a bit of them.
- Release ·
Glyd v0.32.1: Gemma 4 writes faster, glyd fetch, clearer refusals
Glyd v0.32.1: Gemma 4 12B at wren from 81.2 to 88.9 tokens a second on an RTX 4080 SUPER, glyd fetch, and a clear stop where a glyd/ repo does not fit.
- Release ·
Glyd v0.32.0: Gemma 4 12B and 31B, lossless
Glyd v0.32.0: Gemma 4 12B and 31B at :penguin, every weight Google's bf16 weight bit for bit, the 12B on a 24 GB card and the 31B on a 48 GB card.
- Release ·
Glyd v0.31.0: Gemma 4 31B and 12B, exact at 4.5 bits, with drafting
Glyd v0.31.0: Gemma 4 31B and 12B at :wren, Google's QAT weights bit for bit at 4.5 bits per weight, with Google's draft models.
- Release ·
Glyd v0.30.0: download and run, and Nemotron 3 Super on one GPU
Glyd v0.30.0: glyd run MODEL:swift downloads a ready-made checkpoint and starts from it. Nemotron 3 Super (120B) at :hummingbird on one 16 to 24 GB GPU.
- Release ·
Glyd v0.29.4: levels, and a new one, swift
Glyd v0.29.4: glyd run MODEL:LEVEL picks the bits the weights keep. New :swift, about 5.5 bits per weight: Qwen3.8-27B in 17.4 GiB at 47.5 tokens a second.
- Release ·
Glyd v0.29.3: glyd run --lossy, faster and closer to bf16
Glyd v0.29.3: glyd run --lossy at about 6.5 bits per weight, closer to bf16, and Qwen3.8-27B at 31.3 tokens a second on one L40 against FP8's 25.4.
- Release ·
Glyd v0.29.2: Qwen3.8-27B in 21 GiB with glyd run --lossy
Glyd v0.29.2 adds glyd run --lossy: Qwen3.5, 3.6 and 3.8 at about 6.8 bits per weight, 24% less GPU memory for weights than FP8 at its speed.
- Release ·
Glyd v0.29.1: more newer models, less GPU memory
Glyd v0.29.1 improves startup and tool calls for newer model families, compresses more loaded weights, and fixes correctness for mixture-of-experts models.
- Release ·
Glyd v0.29.0: Qwen3.5, and vLLM compiled by default
Glyd v0.29.0 measures Qwen3.5: Qwen3.5-9B's weights 28.2% under bf16's on an RTX 4090. glyd run starts vLLM compiled, and all of Glyd is under BUSL-1.1.
- Release ·
Glyd v0.28.0: the whole model smaller
Glyd v0.28.0 packs the embedding and the output layer under vLLM: Qwen3-8B's weights are 32% under bf16's. Mixtures of experts serve faster than bf16.
- Release ·
Glyd v0.27.0: lossless KV cache in vLLM
Glyd v0.27.0 holds vLLM's KV cache in fewer bits, bit for bit: 1.25 to 1.30 times the tokens on five models. The GPU package now ships compiled.
- Release ·
Glyd v0.26.0: one command to a local chat, serving with vLLM
Glyd v0.26.0 installs with one command and starts a model with glyd run; vLLM serves it packed, now measured on an H100 SXM, and long prompts run faster.
- Release ·
Glyd v0.25.1: long prompts on the L4 and L40S, faster on Hopper
Glyd v0.25.1 brings long prompts on the L4 and L40S within a few percent of bf16's at 8,192 tokens, and makes exact mode on Hopper faster again.
- Release ·
Glyd v0.25.0: glyd pack and verify in Rust, 12-bit saves, a faster 12-bit layout
Glyd v0.25.0 packs and checks a model on the CPU with glyd pack and glyd verify, loads 12-bit saves as saved, and runs the 12-bit layout faster.
- Release ·
Glyd v0.24.0: generate() compiled by default, faster Hopper prompts
Glyd v0.24.0 compiles generate() by default on PyTorch 2.13 or later, and runs Hopper's prompts and an A10's long prompts faster.
- Release ·
Glyd v0.23.0: the GPU kernels as a CUDA library, faster prompts on RTX 40
Glyd v0.23.0 ships its GPU kernels as a CUDA library for engines in C, C++ and Rust, with a C header and example, and runs prompts faster on RTX 40 GPUs.
- Release ·
Glyd v0.22.0: mixtures of experts saved packed, faster on H100 and A100
Glyd v0.22.0 saves and loads mixtures of experts packed, packs Llama 4's experts, and takes less GPU time than bf16 on an H100 up to 64 sequences a step.
- Release ·
pip install "glyd[gpu]": a model 33% smaller on the GPU, every weight exact
Glyd v0.21.0 installs with pip: glyd.from_pretrained() packs a bf16 model on the GPU as it loads, every weight bit for bit; exact=True gives bf16's logits.
- Plans ·
What's next for Glyd: pip install, vLLM, one command
From a PyTorch harness to pip install "glyd[gpu]" with models ready on Hugging Face, then serving through vLLM, then glyd fit, pack and serve. No dates.
- Results ·
How much smaller Glyd makes bf16 weights, bit for bit
Glyd stores bf16 weights in about 11 bits, not 16, and returns every value exactly: every matrix of nineteen open models came out 31.9 to 33.0% smaller.