Vibeleaderboard
← All Intel
Intel / article

Jev for Python engineers

Source
vercel.com
Author
Yury Selivanov
Date

Context

Vercel's AI SDK for Python adds an experimental interface to Jev, a model that answers multiple-choice-style questions about supplied data. Developers define the possible answers, and the model returns its answers with confidence for each, in structured form that application code can use directly. Python integration

This extends the decision-oriented approach in TypeSafe AI's earlier Jev introduction. But a correctly shaped answer can still be wrong: in the Vercel article's experiment distinguishing English from Python, Jev did much better than the author's hand-trained classifier yet still treated a partially typed Python expression as English. The integration makes constrained decisions easy to request; whether those decisions are reliable enough depends on the task, as the article itself says. Experiment

Key takeaways · AI-distilled
  • Vercel describes Jev as a universal classifier: you pass it state plus multiple-choice questions and it returns structured JSON answers with confidence instead of generated text, aiming to be cheap and fast without task-specific training.
  • The Python AI SDK exposes it through one experimental evaluate() call taking a model, state and a mapping of questions: ChoiceQuestion picks one answer, ScoreQuestion rates on a scale you define, and NoulQuestion estimates the probability a statement is true.
  • Used as a live Python-or-English detector for an agentic REPL, Jev did better than the author's hand-trained classifier but still had gaps, labeling "what's" + " up as English even though it is a partially typed Python expression.
  • To make Jev write code, the author had GPT-5.6 expand the prompt into a detailed plan, then had Jev build a Python AST one choice at a time. The result was syntactically valid but mostly incorrect, which the author presents as a limit of using a classifier for generation.
More from Yury Selivanov
Recommended reads
Comments

Checking sign-in…

Loading comments…