Glyd
GitHubGet started

Qwen3.5-9B

  • Qwen
  • 9.7B parameters
  • Dense
  • Apache-2.0
  • Released in bf16
  • Measured Oct 5, 2026

Qwen/Qwen3.5-9B on Hugging Face

Raw logsRun it

Qwen3.5-9B with Glyd

bf16GlydChange, %
Weights in vLLM on an RTX 4090, GiBMeasured, vLLM 0.30.0, v0.29.0log17.6612.68−28log
Weights in vLLM on an RTX 5090, GiBMeasured, vLLM 0.30.0, v0.29.0log17.6613.89−21log
Smallest single GPU, GBWeights, an 8K-token KV cache and 1.5 GB for the runtime2416
H100 80 GB GPUs it takes11

Which single GPU it fits

Worked out from the measured weights.

MemoryGPUsbf16Glyd
16 GBRTX 4080, RTX 5080NoFits
24 GBRTX 4090, RTX 3090, A10FitsFits
32 GBRTX 5090FitsFits
48 GBRTX A6000, L40S, RTX 6000 AdaFitsFits
80 GBH100, A100 80 GBFitsFits
96 GBRTX PRO 6000, GH200FitsFits
141 GBH200FitsFits

Run it

pip install "glyd[gpu]"
import glyd
model = glyd.from_pretrained("Qwen/Qwen3.5-9B")

Linux and an NVIDIA GPU, Ampere or later: the weights are packed on the GPU as the model loads, every one bit for bit (getting started). vLLM serves Glyd's weights too, for the models checked there (serving with vLLM).

Related models

Also measured

Twenty-one more open models measured, with quality wherever bf16 fits one GPU.