Clip transcript
>> Thanks, Ankush. So, let's now look into how semantic routing works. If you have built a rack system previously, this will feel pretty similar to that. The difference is that we will retrieve tools instead of the documents. So, at the first stage, say for example, over here, each tool will have a clear description. For example, a search flights tool, uh a check calendar availability tool, or a retrieve customer order status. Say for example, the second step would be those descriptions needs to get embedded and stored in a vector index. This is usually done offline when the tool catalog is created or updated. At the runtime, the user will ask a question, we embed the user query with the same embedding model. Then, we search that vector index for the tools with the description that are closest to the query. So, the router will then return the top K tools. Often, K would be three or five tools that matches the user query. And only those selected schemas will get injected into the model call. So, this is the whole pattern. At the first step, you need to index tool descriptions offline. Then, retrieve relevant tools at runtime and keep the model's context focused. So, in short, the idea is pretty simple. Semantic routing is basically a rack for tools. Um if you already have an embedding model and vector database in your stack, most of the infrastructure is already familiar to you.