What runs on your GPU

Pick your GPU's memory. Open models in their original precision, no quantization: which fit on one card in bf16, which fit only with Glyd, and how much room is left for the context.

GPU memory	NVIDIA GPUs	Only with Glyd	Either way
		Open models on one card
16 GBRTX 4080, 4080 SUPER, 5080, 5070 Ti and 4060 Ti 16 GB	RTX 4080, 4080 SUPER, 5080, 5070 Ti and 4060 Ti 16 GB	6	7
24 GBRTX 4090, RTX 3090 and A10	RTX 4090, RTX 3090 and A10	4	13
32 GBRTX 5090	RTX 5090	0	17
48 GBRTX A6000, L40S and RTX 6000 Ada	RTX A6000, L40S and RTX 6000 Ada	12	18
80 GBH100 and A100 80 GB	H100 and A100 80 GB	0	30
96 GBRTX PRO 6000 and GH200	RTX PRO 6000 and GH200	2	30
141 GBH200	H200	3	32

The GPUs each release is tested on

Updated Oct 5, 2026. A model fits when its weights, an 8K-token KV cache and 1.5 GB for the runtime fit in the memory nvidia-smi reports.