Building a Voice Agent That Joins Zoom, Meet and Teams Calls

AssemblyAI pointed to a Recall.ai tutorial, written by Aydin Schwartz, that builds a voice agent which joins a Zoom, Google Meet, Microsoft Teams or Webex call and answers participants out loud. The useful idea is that the agent never integrates with a meeting platform directly. It is an ordinary webpage. AssemblyAI's Voice Agent API runs the conversation inside that page: it transcribes incoming speech, generates a reply with a language model, and streams speech back. A Recall.ai meeting bot then joins the call, opens the same page in its own browser, feeds the meeting's audio to the page as if it were a microphone, and sends whatever the page plays and displays back into the meeting, through a feature Recall.ai calls Output Media. Because the bot loads the same page on every platform, the tutorial says no separate implementation is needed for each meeting service.

Much of the configuration deals with the agent being one participant among several. The system prompt tells it to reply only when someone addresses it directly and otherwise to return the word SKIP, which the page detects so that reply is never spoken. The transcription settings add context about the meeting, a list of names and terms the speech model might mishear, and a far-field mode meant for laptop microphones and conference rooms. Tools let the agent fetch information or act during the call. The sample's weather lookup is set to hold the agent's reply until the result returns, so a new remark from someone else does not set off a second answer, and a leave_meeting tool asks Recall.ai to remove the bot when someone tells it to go. Documents uploaded before the call are simply added to the system prompt, which the author says works for a demo but not for hundreds of documents, where a search tool would be the better route.

The page keeps both companies' API keys out of the browser: the server issues a short-lived AssemblyAI token, and the agent's URL carries a short-lived session token so an old link cannot be reused indefinitely. The bot runs on Recall.ai's larger four-core variant because it is doing more work than a notetaker, which the tutorial notes costs extra. This is a vendor walkthrough of two vendors' APIs with sample code, not a measured evaluation, and it reports no latency or accuracy figures.