LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
determinism — Whether the same input reliably produces the same output — something LLM systems mostly lack, which changes how you test and debug them.
Why it matters
JevEval replaces free-form LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition →-judge verdicts with bounded yes/no questions answered as calibrationHow well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.Full definition → probabilities and deterministic scoring math, addressing the determinismWhether the same input reliably produces the same output — something LLM systems mostly lack, which changes how you test and debug them.Full definition → problem of asking a model to just invent a number.
