Glyd
GitHubGet started

Llama 3.3 70B

  • Meta
  • 70.6B parameters
  • Dense
  • Llama 3.3 license
  • Released in bf16
  • Measured Sep 27, 2026

meta-llama/Llama-3.3-70B-Instruct on Hugging Face

Raw logsRun it
Weights in GPU memory
141 to 94.7 GB
33% less, every weight restored bit for bit
Smallest single GPU
141 GB to 96 GB
Weights, an 8K-token KV cache and 1.5 GB for the runtime
H100 80 GB GPUs it takes
2 to 2
Weights, an 8K-token KV cache and 1.5 GB a GPU for the runtime
Its matrices, packed
136.9 to 91.9 GB
−32.9%, every matrix unpacked bit for bit

Which single GPU it fits

Worked out from the measured weights.

MemoryGPUsbf16Glyd
16 GBRTX 4080, RTX 5080NoNo
24 GBRTX 4090, RTX 3090, A10NoNo
32 GBRTX 5090NoNo
48 GBRTX A6000, L40S, RTX 6000 AdaNoNo
80 GBH100, A100 80 GBNoNo
96 GBRTX PRO 6000, GH200Room for about 19K tokens of context with GlydNoFits
141 GBH200Room for the KV cache: about 25K tokens in bf16, 166K with GlydFitsFits

Every matrix, bit for bit

Every Linear layer's matrix packed and unpacked; H100 SXM, Sep 27, 2026.

Matrices in bf16136.9 GB
With Glyd, tiered layout91.9 GB
Change−32.9%

The 12-bit layout takes 24.7 to 24.8% off each matrix and decodes with less work. The ten-model report

Run it

git clone https://github.com/surya-koritala/Glyd
cd Glyd/gpu
python e2e.py /models/Llama-3.3-70B-Instruct --format auto --fused \
  --merge --baseline --batch 1,8,32

Today Glyd runs from its PyTorch harness; vLLM and SGLang integration is not built yet.

Also measured

Twenty more open models measured, with quality wherever bf16 fits one GPU.