What runs on a 80 GB GPU, exactly
H100 and A100 80 GB. Open models in their original precision, no quantization, with and without Glyd.
Runs only with Glyd
Runs either way
| Needs, GB | Room for context, K tokens | |||
|---|---|---|---|---|
| Model | bf16 | Glyd | bf16 | Glyd |
 Qwen3.5-2Best. | 6.3 | 4.8 | 6,458 | 6,580 |
 SmolLM3-3B | 8.4 | 6.3 | 1,055 | 1,082 |
 Llama-3.2-3B-Instruct | 9.0 | 6.9 | 676 | 694 |
| NVIDIA-Nemotron-3-Nano-4B-BF16est. | 9.7 | 7.1 | 4,636 | 4,796 |
| Qwen3-4B-Instruct-2507 | 10.9 | 8.3 | 515 | 532 |
| Qwen3.5-4Best. | 11.2 | 8.1 | 2,276 | 2,370 |
 gemma-4-E2B-itest. | 11.9 | 8.6 | 10,276 | 10,745 |
 Mistral-7B-Instruct-v0.3 | 17.2 | 12.4 | 530 | 566 |
| gemma-4-E4B-itest. | 17.8 | 12.5 | 4,738 | 5,102 |
| Llama-3.1-8B-Instruct | 18.7 | 13.5 | 518 | 558 |
| Qwen3-8B | 19.2 | 14.0 | 458 | 493 |
 MiMo-V2.6-Distill-Qwen-9Best. | 20.7 | 14.5 | 1,986 | 2,174 |
| Qwen3.5-9B | 21.2 | 15.7 | 1,971 | 2,138 |
| gemma-4-12B-itest. | 26.4 | 18.6 | 915 | 1,035 |
| gemma-3-12b-it | 26.9 | 18.8 | 909 | 1,031 |
 phi-4 | 32.6 | 23.0 | 267 | 314 |
 DeepSeek-R1-Distill-Qwen-14B | 32.8 | 23.3 | 277 | 324 |
| Mistral-Small-3.2-24B-Instruct-2506 | 51.0 | 35.1 | 219 | 316 |
| gemma-4-26B-A4B-it | 53.8 | 36.8 | 789 | 1,202 |
| gemma-3-27b-it | 57.6 | 39.6 | 355 | 574 |
| Qwen3.8-27B | 57.7 | 39.5 | 433 | 711 |
| Qwen3.5-27Best. | 57.7 | 39.5 | 433 | 710 |
| Qwen3.6-27Best. | 57.7 | 39.5 | 433 | 710 |
| Muse-Glimmer-30B | 61.4 | 41.8 | 1,830 | 3,297 |
| Qwen3-30B-A3B | 63.5 | 43.5 | 232 | 435 |
| gemma-4-31B-itest. | 66.3 | 45.9 | 130 | 255 |
| Qwen3-32B | 69.3 | 48.3 | 70 | 150 |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16est. | 67.5 | 46.0 | 2,943 | 6,445 |
| Qwen3.5-35B-A3Best. | 73.7 | 50.2 | 586 | 1,734 |
| Qwen3.6-35B-A3Best. | 73.7 | 50.2 | 586 | 1,734 |
Too big for 80 GB
| Needs, GB | |||
|---|---|---|---|
| Model | bf16 | Glyd | Fits on |
| Llama-3.3-70B-Instruct | 145.4 | 99.0 | 96 GB |
| Qwen2.5-72B-Instruct | 149.7 | 102.1 | 96 GB |
| Qwen3-Coder-Nextest. | 161.2 | 109.1 | 141 GB |
| Qwen3-Next-80B-A3B-Instruct | 164.5 | 112.1 | 141 GB |
Speed on this GPU
| GPU time a step, ms | |||
|---|---|---|---|
| Sequences a step | bf16 | Glyd | Change, % |
| 1 | 12.08 | 11.08 | −8 |
| 8 | 13.11 | 12.34 | −6 |
| 32 | 14.26 | 13.53 | −5 |
| 64 | 15.19 | 14.77 | −3 |
| 128 | 17.29 | 17.94 | +4 |
Questions about 80 GB GPUs
Which models run on a 80 GB GPU only with Glyd?
None of the models we track: every one that fits with Glyd fits in bf16 too, and Glyd leaves the difference for a longer context.
Is it slower than bf16?
Usually not, at a few sequences a step: the table on this page gives v0.22.0's numbers. At many sequences, and for long prompts on some GPUs, Glyd still takes longer (the known gaps on the benchmarks page).
Has Glyd been measured on a 80 GB GPU with a 2026 model?
Yes: Qwen3.8-27B's weights took 38.77 GiB with Glyd on an H100 SXM against bf16's 50.22 (vLLM 0.30.0, v0.28.0). The KV cache held 204,117 tokens against bf16's 122,538 (1.67 times).
How is this different from a 4-bit GGUF?
A 4-bit model is smaller, 4 bits a weight and their scales, but its weights are rounded and its answers change. Glyd gives you the original model.