calibration

Accuracy asks "is it right?"; calibration asks "does it know when it's right?" A miscalibrated model is confidently wrong and hesitantly right, which is worse than it sounds: downstream systems and humans act on that stated confidence — auto-approving what the model is "sure" about, escalating what it flags.

For agent builders it's the load-bearing property behind every threshold: when should an agent act, retry, or hand off to a human? Recent work in the index shows calibration is fragile — bolting cost-benefit policies into a judgment prompt can corrupt the probabilities themselves.