Release · Oct 16, 2026 · Glyd
Glyd v0.31.0: Gemma 4 31B and 12B, exact at 4.5 bits, with drafting
Glyd v0.31.0: Gemma 4 31B and 12B at :wren, Google's QAT weights bit for bit at 4.5 bits per weight, with Google's draft models.
Glyd v0.31.0 is out.
Gemma 4 31B and 12B. glyd run glyd/gemma-4-31B-wren downloads glyd/gemma-4-31B-wren (17.4 GB, plus Google’s 1.0 GB draft model) and starts it. The 12B is glyd/gemma-4-12B-wren (6.8 GB + 0.85 GB). Both are made from Google’s QAT checkpoints and keep those weights bit for bit, at 4.5 bits per weight.
curl -LsSf https://getglyd.com/install.sh | sh
glyd run glyd/gemma-4-31B-wrenDrafting. Google’s small Gemma 4 draft model guesses up to 8 tokens and the big one checks them in one pass. Greedy output is the same token for token as without it; sampled output keeps the model’s probabilities. --no-draft turns it off.
One user, greedy, measured:
| GPU, model | Plain | With the draft model, chat prompts | 30K-token prompt | GSM8K | MMLU-Pro |
|---|---|---|---|---|---|
| RTX 4090, 31B | 46 tokens a second | 78-80 | 20 s | 72/75 | 70/75 |
| RTX 4080 SUPER, 12B | 82.5 | 109 | 12.9 s | 72/75 | 66/75 |
On predictable text the draft model does more: up to 176 tokens a second for the 31B and 232 for the 12B. The 31B used 21.8 of the 4090’s 24.6 GB at 32K context.
glyd verify MODEL checks every tensor of a Glyd model against the sha256 recorded when it was made, and says whether the weights are the source’s bit for bit.
glyd verify glyd/gemma-4-31B-wrenKnown limits: Gemma 4 runs on Linux with an NVIDIA RTX 30 series card or newer, text only, one request at a time. Gemma 4 26B-A4B, E4B and E2B aren’t supported yet.
Full details are in the release notes and the changelog; what is not done yet is on the known gaps.