quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
Why it matters
Explains how to reach near-lossless 3.5-bit quantizationShrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.Full definition → by combining LDLQ calibrationHow well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.Full definition → error reduction with full-network self-distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition →, and ships it as an open library that exports directly to llama.cpp datatypes.
