JevEval makes an AI grading rubric explicit in software. Developers choose which test-case information the evaluator can see, write bounded questions about it and assign weights. Jev, TypeSafe AI's decision model, returns probabilities for the allowed answers; the evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → framework converts them into a weighted score using fixed arithmetic. The JevEval announcement shows how this separates the model's judgments from the calculation that combines them. It applies the earlier decision-model approach to evaluation, making the questions and their relative importance inspectable.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
