What runs on a 32 GB GPU, exactly
RTX 5090. Open models in their original precision, no quantization, with and without Glyd.
Runs only with Glyd
Runs either way
| Needs, GB | Room for context, K tokens | |||
|---|---|---|---|---|
| Model | bf16 | Glyd | bf16 | Glyd |
 Qwen3.5-2Best. | 6.3 | 4.8 | 2,281 | 2,402 |
 SmolLM3-3B | 8.4 | 6.3 | 358 | 386 |
 Llama-3.2-3B-Instruct | 9.0 | 6.9 | 228 | 246 |
| NVIDIA-Nemotron-3-Nano-4B-BF16est. | 9.7 | 7.1 | 1,503 | 1,663 |
| Qwen3-4B-Instruct-2507 | 10.9 | 8.3 | 166 | 184 |
| Qwen3.5-4Best. | 11.2 | 8.1 | 710 | 803 |
 gemma-4-E2B-itest. | 11.9 | 8.6 | 3,115 | 3,584 |
 Mistral-7B-Instruct-v0.3 | 17.2 | 12.4 | 138 | 174 |
| gemma-4-E4B-itest. | 17.8 | 12.5 | 1,157 | 1,521 |
| Llama-3.1-8B-Instruct | 18.7 | 13.5 | 126 | 166 |
| Qwen3-8B | 19.2 | 14.0 | 110 | 145 |
 MiMo-V2.6-Distill-Qwen-9Best. | 20.7 | 14.5 | 420 | 608 |
| Qwen3.5-9B | 21.2 | 15.7 | 405 | 571 |
| gemma-4-12B-itest. | 26.4 | 18.6 | 132 | 252 |
| gemma-3-12b-it | 26.9 | 18.8 | 125 | 248 |
 phi-4 | 32.6 | 23.0 | 16 | 63 |
 DeepSeek-R1-Distill-Qwen-14B | 32.8 | 23.3 | 15 | 63 |
Too big for 32 GB
| Needs, GB | |||
|---|---|---|---|
| Model | bf16 | Glyd | Fits on |
| Mistral-Small-3.2-24B-Instruct-2506 | 51.0 | 35.1 | 48 GB |
| gemma-4-26B-A4B-it | 53.8 | 36.8 | 48 GB |
| gemma-3-27b-it | 57.6 | 39.6 | 48 GB |
| Qwen3.8-27B | 57.7 | 39.5 | 48 GB |
Questions about 32 GB GPUs
Which models run on a 32 GB GPU only with Glyd?
None of the models we track: every one that fits with Glyd fits in bf16 too, and Glyd leaves the difference for a longer context.
Is it slower than bf16?
Not measured on a 32 GB GPU yet. In v0.22.0's run, one sequence at a time took 8 to 25% less GPU time a step than bf16 on every GPU tested.
Has Glyd been measured on a 32 GB GPU with a 2026 model?
Yes: Qwen3.5-9B's weights took 13.89 GiB with Glyd on an RTX 5090 against bf16's 17.66 (vLLM 0.30.0, v0.29.0).
How is this different from a 4-bit GGUF?
A 4-bit model is smaller, 4 bits a weight and their scales, but its weights are rounded and its answers change. Glyd gives you the original model.