What's next for Glyd: pip install, vLLM, one command

Update: the first phase shipped in v0.21.0: pip install "glyd[gpu]", with glyd.from_pretrained() packing any bf16 model on the GPU as it loads (the release). The second, serving through vLLM, shipped in v0.26.0: vllm serve MODEL --quantization glyd (serving with vLLM). The third began with it: curl -LsSf https://getglyd.com/install.sh | sh, then glyd run MODEL (local chat, like Ollama), with glyd serve and glyd doctor beside it, and glyd pack and glyd verify since v0.25.0.

The benchmarks on this site were measured with Glyd’s PyTorch harness (getting started). The plan to make Glyd one command, in three phases, with no dates:

1. pip install

pip install "glyd[gpu]"

A package for the GPU part, and models already packed on Hugging Face, so that loading one is a line of Python with glyd.from_pretrained(): the model in about 11 bits a weight on the GPU, every weight its bf16 value, without packing it yourself.

2. Serving through vLLM

vllm serve MODEL --quantization glyd

Serving engines are how models run in production, and today Glyd is not in one: its speed numbers come from the harness, with q, k, v and gate, up merged the way vLLM runs them. The second phase is Glyd as an option of vLLM, so that a model that needs two GPUs in bf16 can be served from one, as Qwen3-32B runs on one 48 GB GPU in the harness.

3. One command

glyd fit
glyd pack
glyd serve

The steps of the harness as commands of their own: whether a model fits your GPU, the question the GPU pages answer from the fit rule; packing it; and serving it.

What stays the same

The model does not change: every weight decodes to exactly the bf16 value it was, so the answers are the model’s, up to a rounding in the last bit of a sum (how close). And every number keeps coming from a run whose log is in the repository: benchmarks/gpu.

Today pip install "glyd[gpu]" runs any model released in bf16. What changes as it happens: the changelog and the roadmap, and this blog’s RSS feed.