Decision models remove training, not production ML Engineering
- Source
- agentunicorn.ai
- Author
- flashnik
- Date

Context
Nikita Pustovoytov, founder of the voice-AI company AgentUnicorn, argues in an essay that reusable "decision models" remove the cost of training a model for each decision but not the cost of validating it. In the essay's operational description, you give such a model an application's state and a bounded question, such as whether a support chat should escalate to a human, receive probabilities, and map them to an action. The essay uses Jev, which it attributes to TypeSafe AI, as its example.
What remains, the essay says, is testing on representative traffic, measuring error at the confidence threshold actually used, and naming an accountable owner for each decision. Its illustration: if 95 percent of support requests should continue and 5 percent should escalate, a policy that always answers "continue" scores 95 percent accuracy while catching no escalations. It also uses the rule of three: with zero failures in n tests, the 95 percent upper bound on the error rate is roughly 3 divided by n, about 1 percent at 300 cases. These are one practitioner's arguments, not experimental results.
- calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
Checking sign-in…
Loading comments…



