inference — Model Foundations

Jul 14, 2026 quantization / inference / llm

SpinQuant, when rotations are worth learning

Zechun Liu et al., 2024 — ICLR 2025

QuIP proved random rotations make models quantizable. SpinQuant asks the obvious next question: why random? It parameterizes the rotations, descends the quantized network's own loss on the orthogonal manifold for 100 steps, and the answer is decisive, especially at 70B where random rotations can collapse. This is also the method Meta actually ships on-device. I audited the repo, verified the manifold math numerically, and reproduced learned-beats-random in miniature.

repro: partial 6 verified

Jul 05, 2026 quantization / inference / llm

GPTQ, with the code open

Elias Frantar et al., 2022 — ICLR 2023

GPTQ shrinks a trained model's weights from 16 bits to 3 or 4 bits while barely changing what the model computes. I read the paper and its 171-line reference implementation side by side, and wrote this up so that it needs no prior background in quantization. The math is built up from small numerical examples.

repro: partial 2 verified1 partial1 unverified1 refuted