Intel / article

Decision Models in Hugging Face

Source: huggingface.co
Date: Published Nov 10, 2026

Context

Hugging Face, the hosting site for open AI models that also maintains the Transformers library, has added a standard way to run decision models across its tools. A decision model is given some input, called the state, plus a set of named questions, each with answers the application has already defined, and it returns a probability for each allowed answer instead of writing text. The post starts from TypeSafe AI, which launched Jev in September and called it a System One model, and describes the field as young and fast-moving, with models built in quite different ways. Hugging Face's aim is a shared request and response format, so any of these models can be served, found and fine-tuned the same way however it works inside.

The post's support-ticket example shows the difference from asking a chatbot. To route the message "I was charged twice for my order and want a refund," a developer can prompt a to return JSON, but the model may produce invalid JSON or an answer outside the allowed list. , which blocks tokens that would break the format, fixes the structure but still produces the answer one token at a time. A decision model scores the allowed answers directly; in the post's server response the request used 148 input tokens and 0 output tokens. One request can carry several questions of three types. A choice question picks among options and returned billing with probability 0.998. A score question rates an ordered scale; with low, medium and high numbered 0 to 2, it reports the probability-weighted average, here 1.7 to 1.75 across the post's examples, leaning toward high. A noul question (named after the Bernoulli distribution) returns the probability that a yes-or-no statement is true, 0.995 for "is the customer requesting a refund?". Choice and score answers also carry a confidence figure; for a choice it rescales the top probability so that an even split counts as 0 and certainty as 1.

The software pieces are concrete. Transformers adds a decision pipeline and model classes, and its local server and llama.cpp, the popular engine for running models on ordinary hardware, both accept the same request at an endpoint named /v1/systemone. The Hub has a decision-model tag for finding compatible models, the post shows one with Transformers' standard trainer, and dedicated support in TRL, Hugging Face's training library, is an open pull request. Transformers can also run the interface on compatible ordinary generative models by reading the scores of the answer labels in a single pass, so a team can try the format before adopting a specialized model.

The post standardizes an interface; it does not measure whether decision models are more accurate or cheaper than prompting a large model, and its mention of decisions is described as a possibility of specialized training, not a result. The name is also contested. Hugging Face says the community has settled on decision model, while TypeSafe argued a week earlier, in Don't Call It a Decision Model, that Jev makes small judgments and the application's code makes the decisions. The endpoint path keeps TypeSafe's term.

Key takeaways · AI-distilled

Hugging Face describes decision models as taking a state plus named questions with predefined answers and returning probabilities over those answers, instead of generating text that must then be parsed as JSON.
The interface has three answer types: choice (a distribution over options), score (the expected value over an ordered scale, so 1.7 means belief leaning high, not a level between medium and high), and noul (the probability a yes/no statement is true).
Transformers adds a decision pipeline, AutoModelForDecision and a /v1/systemone endpoint. Per the post, the API also works on compatible autoregressive LLMs by scoring answer-label logits in one batched forward pass, without generating response text.
llama.cpp serves decision models through the same /v1/systemone API, and the Hub adds a decision-model tag, so ModernBERT-based encoders and LLM-derived models can sit behind one contract regardless of architecture.
The authors frame this as an interface rather than a new capability: zero-shot classifiers already allowed runtime labels, and decision models add several typed questions per request. TRL training support was still an open PR at publication.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
structured output — Forcing a model's response to match a schema, so downstream code can parse it instead of guessing at prose.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.

Read the source huggingface.co