Reranking 30 candidates with Jev lifted categorization to 97.2%
- Source
- x.com
- Date
Channel3 reported how it categorizes products with Jev, a decision model from TypeSafe AI. Its first attempt walked the product taxonomy from the top down with beam search, and that approach failed. The approach that worked used embeddings to prefilter each product to 30 candidate categories, then had Jev choose among them. Accuracy rose from 81.9% to 97.2% at a similar cost. The write-up also notes cases where cheap batched models match Jev's results. Classification against a large label set shows up in many agent pipelines, from routing to tagging, so the pattern of a cheap retrieval step followed by a focused decision step fits those steps as well as product catalogs.
Now that the dust has settled with Jev, here's where it works well and where it falls short https://t.co/XRs21utJM0
- reranking — A second pass that re-scores retrieved candidates by reading each one against the query, fixing the ordering that fast vector search got approximately right.
- embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Checking sign-in…
Loading comments…