What runs on a 141 GB GPU, exactly
H200. Open models in their original precision, no quantization, with and without Glyd.
Verified bit for bit, and timed against bf16 in the same run, on the RTX 4090, RTX 5090, H100 SXM, H100 PCIe, A100, A10, A10G, RTX PRO 6000 and RTX 4080 SUPER.
See the benchmarksRuns only with Glyd
| Needs, GB | Room for context, K tokens | ||
|---|---|---|---|
| Model | bf16 | Glyd | Glyd |
 Qwen3-Coder-Nextest. | 161.2 | 109.1 | 1,705 |
| Qwen3-Next-80B-A3B-Instruct | 164.5 | 112.1 | 1,581 |
 Llama-4-Scout-17B-16E-Instruct | 220.5 | 149.0 | 17 |
Runs either way
| Needs, GB | Room for context, K tokens | |||
|---|---|---|---|---|
| Model | bf16 | Glyd | bf16 | Glyd |
| Qwen3.5-2Best. | 6.3 | 4.8 | 11,767 | 11,888 |
 SmolLM3-3B | 8.4 | 6.3 | 1,939 | 1,967 |
| Llama-3.2-3B-Instruct | 9.0 | 6.9 | 1,244 | 1,263 |
| NVIDIA-Nemotron-3-Nano-4B-BF16est. | 9.7 | 7.1 | 8,618 | 8,777 |
| Qwen3-4B-Instruct-2507 | 10.9 | 8.3 | 957 | 974 |
| Qwen3.5-4Best. | 11.2 | 8.1 | 4,267 | 4,360 |
 gemma-4-E2B-itest. | 11.9 | 8.6 | 19,377 | 19,846 |
 Mistral-7B-Instruct-v0.3 | 17.2 | 12.4 | 1,027 | 1,063 |
| gemma-4-E4B-itest. | 17.8 | 12.5 | 9,288 | 9,652 |
| Llama-3.1-8B-Instruct | 18.7 | 13.5 | 1,015 | 1,056 |
| Qwen3-8B | 19.2 | 14.0 | 900 | 936 |
 MiMo-V2.6-Distill-Qwen-9Best. | 20.7 | 14.5 | 3,977 | 4,165 |
| Qwen3.5-9B | 21.2 | 15.7 | 3,962 | 4,129 |
| gemma-4-12B-itest. | 26.4 | 18.6 | 1,911 | 2,030 |
| gemma-3-12b-it | 26.9 | 18.8 | 1,904 | 2,026 |
 phi-4 | 32.6 | 23.0 | 585 | 632 |
 DeepSeek-R1-Distill-Qwen-14B | 32.8 | 23.3 | 608 | 656 |
| Mistral-Small-3.2-24B-Instruct-2506 | 51.0 | 35.1 | 617 | 714 |
| gemma-4-26B-A4B-it | 53.8 | 36.8 | 2,381 | 2,795 |
| gemma-3-27b-it | 57.6 | 39.6 | 1,151 | 1,371 |
| Qwen3.8-27B | 57.7 | 39.5 | 1,428 | 1,706 |
| Qwen3.5-27Best. | 57.7 | 39.5 | 1,428 | 1,705 |
| Qwen3.6-27Best. | 57.7 | 39.5 | 1,428 | 1,705 |
| Muse-Glimmer-30B | 61.4 | 41.8 | 6,730 | 8,197 |
| Qwen3-30B-A3B | 63.5 | 43.5 | 896 | 1,099 |
| gemma-4-31B-itest. | 66.3 | 45.9 | 529 | 653 |
| Qwen3-32B | 69.3 | 48.3 | 319 | 399 |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16est. | 67.5 | 46.0 | 13,560 | 17,063 |
| Qwen3.5-35B-A3Best. | 73.7 | 50.2 | 3,772 | 4,919 |
| Qwen3.6-35B-A3Best. | 73.7 | 50.2 | 3,772 | 4,919 |
| Llama-3.3-70B-Instruct | 145.4 | 99.0 | 25 | 166 |
| Qwen2.5-72B-Instruct | 149.7 | 102.1 | 11 | 157 |
Too big for 141 GB
| Needs, GB | |||
|---|---|---|---|
| Model | bf16 | Glyd | Fits on |
 GLM-4.5-Air | 224.1 | 151.2 | 2 H100s |
| Qwen3.5-122B-A10Best. | 252.0 | 170.2 | 2 H100s |
Questions about 141 GB GPUs
Which models run on a 141 GB GPU only with Glyd?
Qwen3-Coder-Next, Qwen3-Next-80B-A3B-Instruct and Llama-4-Scout-17B-16E-Instruct. With an 8K-token context they need 161.2 to 220.5 GB in bf16, more than one 141 GB GPU holds, and 109.1 to 149 GB with Glyd, every weight exact.
Is it slower than bf16?
Not measured on a 141 GB GPU yet. In v0.22.0's run, one sequence at a time took 8 to 25% less GPU time a step than bf16 on every GPU tested.
How is this different from a 4-bit GGUF?
A 4-bit model is smaller, 4 bits a weight and their scales, but its weights are rounded and its answers change. Glyd gives you the original model.