Oh My Pi adds Drex as a calibrated judge model
- Source
- x.com
- Date
Drex is now in Oh My Pi, the open-source terminal coding agent by @stencil_labs Oh My Pi has a judge role, a model that answers small typed questions instead of writing text. You can now set that role to Drex. Drex support was contributed by the community. Thank you, @devops199fan!
Context
Oh My Pi is an open-source coding that runs in the terminal, built by Stencil Labs, and it lets users assign different models to different jobs. Besides the model that writes code, it keeps a "judge" for the small yes-or-no and pick-one questions the agent needs answered along the way. According to the project's changelog, the judge checks whether a passage found by the agent's search tool actually matches the search, ranks candidate files, and, in opt-in features, decides whether a stored memory is relevant to the current task or whether a session's title still fits. Nace AI says that role can now be set to its Drex models. The changelog shows three Drex models, including Drex Max, which can also judge images, arriving in version 18.20.0 on November 12 from a community contributor.
Drex is what Nace calls a decision model. Instead of writing a reply that the agent then has to parse, it takes a , a question and a list of options, and returns a probability for every option in one step. It cannot answer outside the options it was given. Nace's technical report argues that this is faster and cheaper than asking a large chat model, and reports Drex Max answering a typical request in 183 milliseconds, against about two seconds for GPT-6 Sol. Those are Nace's own measurements.
The probabilities matter because Oh My Pi acts on them. Its search tool reports a passage only when the judge's probability clears a cutoff, 0.2 by default, and the same release added a setting to change that cutoff so a judge other than TypeSafe's Jev can run at the level "that fits its ". A calibrated model is one whose stated confidence matches how often it is right. Nace says that when Drex Max is at least 93% sure, it is right 97% of the time. Artificial Analysis, whose Decision Index v2 scores calibrated confidence as well as correct answers, put Drex Max v1.1 at 73, behind two chat models, Claude Haiku 5.5 and GPT-6 Luna, and within a point of two other decision models. Neither Nace nor the Oh My Pi project has published how Drex performs as Oh My Pi's judge, and the demo video attached to Nace's post could not be reviewed.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
Checking sign-in…
Loading comments…




