What runs on a 48 GB GPU, exactly
RTX A6000, L40S and RTX 6000 Ada. Open models in their original precision, no quantization, with and without Glyd.
Verified bit for bit, and timed against bf16 in the same run, on the RTX 4090, RTX 5090, H100 SXM, H100 PCIe, A100, A10, A10G, RTX PRO 6000 and RTX 4080 SUPER.
See the benchmarksRuns only with Glyd
| Needs, GB | Room for context, K tokens | ||
|---|---|---|---|
| Model | bf16 | Glyd | Glyd |
 gemma-4-26B-A4B-it | 53.8 | 36.8 | 372 |
| gemma-3-27b-it | 57.6 | 39.6 | 159 |
 Qwen3.8-27B | 57.7 | 39.5 | 192 |
| Qwen3.5-27Best. | 57.7 | 39.5 | 191 |
| Qwen3.6-27Best. | 57.7 | 39.5 | 191 |
 Muse-Glimmer-30B | 61.4 | 41.8 | 743 |
| Qwen3-30B-A3B | 63.5 | 43.5 | 90 |
| gemma-4-31B-itest. | 66.3 | 45.9 | 48 |
| Qwen3-32B | 69.3 | 48.3 | 21 |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16est. | 67.5 | 46.0 | 913 |
| Qwen3.5-35B-A3Best. | 73.7 | 50.2 | 74 |
| Qwen3.6-35B-A3Best. | 73.7 | 50.2 | 74 |
Runs either way
| Needs, GB | Room for context, K tokens | |||
|---|---|---|---|---|
| Model | bf16 | Glyd | bf16 | Glyd |
| Qwen3.5-2Best. | 6.3 | 4.8 | 3,692 | 3,813 |
 SmolLM3-3B | 8.4 | 6.3 | 594 | 621 |
| Llama-3.2-3B-Instruct | 9.0 | 6.9 | 379 | 398 |
| NVIDIA-Nemotron-3-Nano-4B-BF16est. | 9.7 | 7.1 | 2,561 | 2,721 |
| Qwen3-4B-Instruct-2507 | 10.9 | 8.3 | 284 | 302 |
| Qwen3.5-4Best. | 11.2 | 8.1 | 1,239 | 1,332 |
| gemma-4-E2B-itest. | 11.9 | 8.6 | 5,534 | 6,003 |
 Mistral-7B-Instruct-v0.3 | 17.2 | 12.4 | 270 | 306 |
| gemma-4-E4B-itest. | 17.8 | 12.5 | 2,367 | 2,731 |
| Llama-3.1-8B-Instruct | 18.7 | 13.5 | 258 | 299 |
| Qwen3-8B | 19.2 | 14.0 | 227 | 263 |
 MiMo-V2.6-Distill-Qwen-9Best. | 20.7 | 14.5 | 949 | 1,137 |
| Qwen3.5-9B | 21.2 | 15.7 | 934 | 1,100 |
| gemma-4-12B-itest. | 26.4 | 18.6 | 397 | 516 |
| gemma-3-12b-it | 26.9 | 18.8 | 390 | 512 |
 phi-4 | 32.6 | 23.0 | 101 | 148 |
 DeepSeek-R1-Distill-Qwen-14B | 32.8 | 23.3 | 104 | 152 |
| Mistral-Small-3.2-24B-Instruct-2506 | 51.0 | 35.1 | 12 | 108 |
Too big for 48 GB
| Needs, GB | |||
|---|---|---|---|
| Model | bf16 | Glyd | Fits on |
| Llama-3.3-70B-Instruct | 145.4 | 99.0 | 96 GB |
| Qwen2.5-72B-Instruct | 149.7 | 102.1 | 96 GB |
| Qwen3-Coder-Nextest. | 161.2 | 109.1 | 141 GB |
| Qwen3-Next-80B-A3B-Instruct | 164.5 | 112.1 | 141 GB |
Questions about 48 GB GPUs
Which models run on a 48 GB GPU only with Glyd?
gemma-4-26B-A4B-it, gemma-3-27b-it, Qwen3.8-27B, Qwen3.5-27B, Qwen3.6-27B, Muse-Glimmer-30B, Qwen3-30B-A3B, gemma-4-31B-it, Qwen3-32B, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, Qwen3.5-35B-A3B and Qwen3.6-35B-A3B. With an 8K-token context they need 53.8 to 73.7 GB in bf16, more than one 48 GB GPU holds, and 36.8 to 50.2 GB with Glyd, every weight exact.
Is it slower than bf16?
Not at 1 to 32 sequences a step: on RTX A6000s, Qwen3-32B on one GPU with Glyd generated 5 to 28% more tokens a second than bf16 across two. At 64, 18% fewer. Where Glyd still takes longer: the known gaps on the benchmarks page.
How is this different from a 4-bit GGUF?
A 4-bit model is smaller, 4 bits a weight and their scales, but its weights are rounded and its answers change. Glyd gives you the original model.