For closed-set decisions like triage or escalation, a typed decision model returned probabilities at a fraction of LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → cost and latency, with nothing to parse. Shows when to route with it and keep the LLM for prose.
Key takeaways · AI-distilled
In OpenRouter's triage run, Claude Opus cost $2.88 per 1,000 tickets at a 1,957 ms median, GPT Luna $0.0921 at 1,106 ms, and Jev $0.0248 at 194 ms. The authors note these are small test sets.
The post lists Jev 1.13 weaknesses: it reads criteria literally, loses accuracy when the state includes irrelevant detail, and is unreliable at arithmetic, counting and date comparison.
Jev 1.13 takes text only (strings, JSON, arrays), with a 64k-tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → cap per request, of which 32k covers the state plus the longest question.
For fairness, the LLMs received the same definitions and escalation rule as Jev in a system promptThe standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.Full definition →, ran at temperature 0, and were asked for a bare JSON object, one request per example.
Jev returns the escalation decision as a single noul probability (0.04 in the sample ticket) that your code compares against a threshold you choose, rather than a verdict to parse.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
