Filtering 151 tools down to 5 with a 350M retrieval model
- Source
- x.com
- Date
Storing too many tools in your context window increases latency and can lead to wrong tool selection. In this demo, we used LFM2.5-ColBERT-350M as a filter to only select the five most relevant tools among 151 options. It's fast and reliable, even without any specific fine-tuning. Try the demo on @huggingface! https://t.co/XhAMqTDzKT And learn more on our blog:
Context
Liquid AI describes using a small retrieval model to shortlist five tools from a collection of 151 before a chooses what to call. The shortlist reduces how many tool descriptions the language model must consider. Demo announcement.
LFM2.5-ColBERT-350M compares a request with candidate text through detailed, per-token matching, as explained in the retriever release. This applies the same retrieval approach used for document search to finding relevant tools. The filter produces a shortlist; the announcement supplies no measured error rate showing how often the necessary tool survives that cut.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
- fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
postLiquid AI ships LFM2.5-24B-A2B open weights for local agents
postToday we release Open d1: two open-weight multimodal models in our d1 decision…
postLiquid AI's d1 decision model adds vision at a fraction of frontier cost
articled1 is now available on @vercel AI gateway!We worked with the amazing vercel…
Checking sign-in…
Loading comments…