Qwen3-32B

Qwen
32.8B parameters
Dense
Apache-2.0
Released in bf16
Measured Sep 26, 2026

	bf16	Glyd	Change, %
Weights in GPU memory, GBEvery weight restored bit for bitlog	65.5	44.5	−32	log
48 GB GPUs it takesRTX A6000log	2	1		log
Tokens per second, 1 sequencebf16 on two A6000s, Glyd on onelog	9.5	11.8	+24	log
MMLU, 1,000 questions, %log	78.5	78.0		log

Memory on RTX A6000s, 48 GB each

nvidia-smi with the model loaded, 49,140 MiB a GPU.

GPU	bf16	Glyd
	In use, MiB
GPU 0	44,554	43,338
GPU 1	18,514	—

bf16 spills onto a second GPU; Glyd runs on one, 5,802 MiB to spare. log

Which single GPU it fits

Worked out from the measured weights.

Memory	GPUs	bf16	Glyd
16 GB	RTX 4080, RTX 5080	No	No
24 GB	RTX 4090, RTX 3090, A10	No	No
32 GB	RTX 5090	No	No
48 GB	RTX A6000, L40S, RTX 6000 Ada	No	Fits
80 GB	H100, A100 80 GB	Fits	Fits
96 GB	RTX PRO 6000, GH200	Fits	Fits
141 GB	H200	Fits	Fits

Tokens per second, RTX A6000

Higher is better. bf16 on two GPUs, Glyd on one.

Sequences a step	bf16	Glyd	Change, %
	Tokens/s
1	9.5	11.8	+24
8	74.4	94.9	+28
32	275.9	289.9	+5
64	488.4	400.5	−18

log

GPU time a token, H100 SXM

Lower is better. q, k, v and gate, up merged, as vLLM runs them.

Sequences a step	bf16	Glyd	Change, %
	GPU time a token, ms
1	27.90	24.50	−12
8	29.78	26.75	−10
32	31.44	33.46	+6
64	33.52	37.35	+11

log

Run it

pip install "glyd[gpu]"

import glyd
model = glyd.from_pretrained("Qwen/Qwen3-32B")

Linux and an NVIDIA GPU, Ampere or later: the weights are packed on the GPU as the model loads, every one bit for bit (getting started). vLLM serves Glyd's weights too, for the models checked there (serving with vLLM).

Related models

Also measured

Twenty-one more open models measured, with quality wherever bf16 fits one GPU.