Glyd
GitHubGet started
MeasuredQwen3-32B on one 48 GB GPU instead of twoQwen3-32B on one 48 GB GPU

Lossless AI compression: 33% less GPU memory, bit for bit.

Glyd stores the weights of open models like Llama, Qwen, Gemma and Mistral in about 11 bits instead of 16, and decodes them inside the GPU's matrix multiply. Every weight comes back exactly, so you run the same model on less hardware, and often faster.

Glyd stores the weights of open models in about 11 bits instead of 16 and decodes them inside the GPU's matrix multiply. The same model on less hardware, often faster.

Check your GPUHow it works
Qwen3-32B on an RTX A6000 (48 GB)Weights as Hugging Face lists them; a 48 GB GPU holds 51.5 GB. Run of Sep 27, 2026.
bf16
bf16, 65.5 GB: needs a second GPU
65.5 GBNeeds a second GPU
Glyd
Glyd, 44.5 GB: fits on one
44.5 GBFits: 43,338 of 49,140 MiB in use

The black line is the GPU's 48 GB. nvidia-smi, Sep 27, 2026.

32 to 33%smaller on every model measured, from SmolLM3 3B to Llama 3.3 70B.smaller on every model measured
Bit for bitEvery weight decodes to its exact bf16 value. Perplexity within 0.04% of bf16.every weight decodes to its exact value
3.4×the speed of DFloat11 at the same size, one sequence, Qwen3-8B on an RTX 4080 SUPER.DFloat11’s speed at one sequence
Up to 28%less GPU time a token than bf16: Qwen2.5-7B, 32 sequences, RTX 4080 SUPER.less GPU time a token than bf16

Will it fit on your GPU?

Pick your GPU's memory. Weights plus an 8K-token KV cache and 1.5 GB for the runtime, bf16 against Glyd.

Weights, an 8K-token KV cache and the runtime, bf16 against Glyd.

Updated Sep 27, 2026
Modelbf16GlydMemory it needs, against this GPUOn one 24 GB GPU
Gemma 3 12B24.4 GB16.4 GBFits only with Glyd
Phi-4 14B29.3 GB19.7 GBFits only with Glyd
R1 Distill Qwen 14B29.5 GB20.1 GBFits only with Glyd
Qwen3 8B16.4 GB11.2 GBFits either way
Llama 3.1 8B16.1 GB10.8 GBFits either way
Mistral 7B v0.314.5 GB9.8 GBFits either way
Mistral Small 3.2 24B48.0 GB32.2 GBToo big for one
Gemma 4 26B-A4B51.6 GB34.7 GBToo big for one
bf16Glydthis GPUSizes measured; est. means worked out from the parameter count, run queued.All models

Smaller, and usually faster

Generating a token reads every weight once, so reading a third fewer bytes saves time. At many sequences a step on A100 and H100 the decode work shows, and Glyd is slower there today.

All benchmarks
GPU time a token against bf16, Qwen2.5-7B1 sequence32 sequences
RTX 4080 SUPER25% less time28% less time
A1028% less time18% less time
A10017% less time6% more time
H100 SXM5% less time8% more time

Decode steps only, q, k, v and gate, up merged as vLLM runs them. Sep 27, 2026.

How it works, in three stepsHow it works

The full explanation
  1. 01

    Find the waste

    A bf16 weight spends 8 bits on its exponent, which carries about 2.6 bits of information in a trained model.A bf16 weight spends 8 bits on its exponent, which carries about 2.6 bits of information.

  2. 02

    Code it exactly

    Common exponents get short codes and rare ones a side list. Sign and mantissa stay as they are: about 11 bits a weight, nothing rounded.Short codes for common exponents, a side list for rare ones. Nothing rounded.

  3. 03

    Decode in the multiply

    The GPU reads the compressed bytes and decodes them in registers, straight into the tensor cores. No bf16 copy is ever made.Decoded in registers, straight into the tensor cores. No bf16 copy in memory.

The full explanation

How it compares

Lossless weight compression is an active research area. Here is where Glyd sits, on numbers we measured ourselves where we could.

MethodBits a weightSame model?Where the weights are decoded
GlydOpen source, 202610.8 or 12.0Yes, bit for bitInside the matrix multiply, in registers. 3.4× DFloat11 at one sequence.
DFloat11NeurIPS 2025about 11Yes, bit for bitEach block decompressed to bf16 in memory before it runs.
ZipServASPLOS 202611.35Yes, bit for bitInside the matrix multiply. Its kernels and Glyd’s are within a few percent.
4-bit quantizationGGUF Q4, AWQ, GPTQabout 4.5No, roundedSmallest by far, and a different model: its answers change.

Try it on your model

Point the harness at any model released in bf16. It packs the weights, checks every one bit for bit, and runs bf16 and Glyd side by side on your GPU.

Check your GPUView on GitHub
git clone https://github.com/surya-koritala/Glyd
cd Glyd/gpu
python e2e.py /models/Llama-3.1-8B-Instruct \
  --format auto --fused --baseline --batch 1,8,32