What runs on a 16 GB GPU, exactly

RTX 4080, 4080 SUPER, 5080, 5070 Ti and 4060 Ti 16 GB. Open models in their original precision, no quantization, with and without Glyd.

Qwen3-8B on an RTX 4080 SUPER, one sequence

Weights	GB	Tokens/s
bf16	16.38	Does not fit
DFloat11	11.16	13.8
Glyd	11.15	47.2

Against DFloat11

Runs only with Glyd

bf16 does not fit; Glyd does, bit for bit

Model	bf16	Glyd	Glyd
	Needs, GB		Room for context, K tokens
Mistral-7B-Instruct-v0.3	17.2	12.4	44
gemma-4-E4B-itest.	17.8	12.5	334
Llama-3.1-8B-Instruct	18.7	13.5	36
Qwen3-8B	19.2	14.0	30
MiMo-V2.6-Distill-Qwen-9Best.	20.7	14.5	89
Qwen3.5-9B	21.2	15.7	52

Runs either way

Glyd leaves room for a longer context or more users

Model	bf16	Glyd	bf16	Glyd
	Needs, GB		Room for context, K tokens
Qwen3.5-2Best.	6.3	4.8	896	1,017
SmolLM3-3B	8.4	6.3	128	155
Llama-3.2-3B-Instruct	9.0	6.9	80	98
NVIDIA-Nemotron-3-Nano-4B-BF16est.	9.7	7.1	465	624
Qwen3-4B-Instruct-2507	10.9	8.3	51	69
Qwen3.5-4Best.	11.2	8.1	190	284
gemma-4-E2B-itest.	11.9	8.6	741	1,210

Too big for 16 GB

Even compressed

Model	bf16	Glyd	Fits on
	Needs, GB
gemma-4-12B-itest.	26.4	18.6	24 GB
gemma-3-12b-it	26.9	18.8	24 GB
phi-4	32.6	23.0	24 GB
DeepSeek-R1-Distill-Qwen-14B	32.8	23.3	24 GB

Speed on this GPU

RTX 4080 SUPER, Qwen3-4B-Instruct-2507, smallest layout, v0.22.0: GPU time a step after the prompt; lower is better. log

Sequences a step	bf16	Glyd	Change, %
	GPU time a step, ms
1	14.55	11.75	−19
8	15.77	12.58	−20
32	17.59	14.24	−19
64	19.39	20.33	+5

Every GPU tested, release by release

Questions about 16 GB GPUs

Can I run an 8B model on a 16 GB GPU without quantizing it?

Yes, with Glyd. Llama-3.1-8B-Instruct and Qwen3-8B take about 16 GB in bf16, which leaves no room to run; Glyd holds them in about 11 GB with every weight exact.

Is it slower than bf16?

Usually not, at a few sequences a step: the table on this page gives v0.22.0's numbers. At many sequences, and for long prompts on some GPUs, Glyd still takes longer (the known gaps on the benchmarks page).

How is this different from a 4-bit GGUF?

A 4-bit model is smaller, 4 bits a weight and their scales, but its weights are rounded and its answers change. Glyd gives you the original model.

Updated Oct 5, 2026. A model fits when its weights, an 8K-token KV cache and 1.5 GB for the runtime fit in the memory nvidia-smi reports: 16,376 MiB on a 16 GB card.