← All IntelClip / AI AgentsThree-step implementation and its true runtime cost
From The 100-Tool Agent Is a Trap - Sohail Shaikh & Ankush Rastogi, Prosodica · ≈16:00
Concrete build recipe — offline catalog and embeddings in any vector store, nearest-neighbor top-K per request, log which tool was selected so K and descriptions can be tuned.
What’s in it
- Concrete build recipe — offline catalog and embeddings in any vector store, nearest-neighbor top-K per request, log which tool was selected so K and descriptions can be tuned.
Clip transcript
discussing about the implementation pattern. There are three steps in it. Step one is offline. So, build a catalog of tool. For each tool, store the name, description, schema. Then, embed the description and store it into a vector database. You can use Chroma DB, you can use Pinecone, you can use Quadrant. Whichever vector database you are using. Then, step two will happen on every request. Embed the user's query. Then, run the nearest neighbor search. And return the top K tools. In the step three, it will also happen on every request. Fetch the schema for those selected tools. And then, pass only those schemas into the model call and log which tool was selected. So, here, logging really matters. So, let's say if you miss something and if you want to improve the description or you want to tune the K, you need a good logging system that you can use it. So, the runtime overhead is small, one embedding call, and one vector search. The payoff is much smaller prompt and more stable tool selection problem.
Comments
Sign in to comment.
Loading comments…