Intel / article

Frontier Model Cost and Open-Weights Popularity is Driving Demand for Model Routing

Source: latent.space
Author: Richard MacManus
Date: Published Aug 18, 2026

Context

Glean, an enterprise AI platform, describes as a decision that improves through feedback from actual work. Employees can choose a model, administrators can restrict the choices, and an automatic mode selects one for each task. Its co-founder Arvind Jain says the company watches when users move to another model after an unsatisfactory result. Interview account.

Jain also says Glean reruns a small fraction of tasks behind the scenes with alternative models, some cheaper and some more expensive, then uses AI-based judges to assess how well the router chose. This gives the router evidence about whether another option would have served a task better. The feedback loop explains how routing can adapt to a company's workload instead of relying only on general model rankings. Evaluation process.

Terms in this piece · Glossary

model routing — Sending each request to a model chosen by the difficulty of the task, rather than using one model for everything.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Read the source www.latent.space