Docs
Glyd has two parts. The one this site is about keeps a model’s bf16 weights compressed in GPU memory, about 11 bits a weight instead of 16, and decodes them inside the matrix multiply: the model is the same to the bit, on less hardware. It lives in the repository’s gpu/ folder and runs today from a PyTorch harness. Underneath it is the codec: a lossless compression library and command line, written in Rust with a C ABI, that installs with Homebrew, cargo or a Python wheel.
The pages
- Getting started: set up the harness on a Linux machine with an NVIDIA GPU, pack a model and run it beside bf16, read what it prints; install the codec.
- On the GPU: the two layouts, which GPU runs which, every option of
e2e.py, and how to repeat a benchmark from this site. - The codec: its levels and modes, the object store, and the Rust, C, Python and Go interfaces.
- FAQ: quantization, accuracy, GPUs, speed, installing.
What is not there yet
A Python package for the GPU part, pip install "glyd[gpu]" with glyd.from_pretrained(), and serving through vLLM are in progress: what’s next. Until then, everything on this site was measured with the harness, and every run’s log is in the repository: benchmarks/gpu.