Vibeleaderboard
← All Intel
Intel / post

Bringing a Billy Bass to Life with AI

Source
x.com
Date
M
mtw14

back in july, I put an ai agent in a big mouth billy bass bringing billai to life. some of you asked how (or why). well, now that bidirectional streaming is GA in strands, I wrote up a deep dive on the the whole thing 🐟 https://t.co/zaltI19uVq

Context

BillAI Bass is a joke project with a serious lesson about voice agents: a conversation feels natural only when the plumbing that moves audio between the speaker, the model and the microphone is right. In the builder's write-up, a Raspberry Pi 5 goes inside a Big Mouth Billy Bass, the motorized singing toy fish, and turns it into a voice assistant that can check the weather, read the news and look at the builder's calendar while its mouth, head and tail move as it talks.

The core technique is bidirectional . Instead of recording a whole question and sending it once the user stops talking, the app streams audio to the model while the person is still speaking. The model decides when the speaker has finished, and its spoken reply streams back over the same connection, so playback starts as the audio arrives rather than after an awkward pause. The build uses BidiAgent from the Strands SDK, a component that runs this loop and executes tool calls, with Amazon's Nova 2.5 Sonic speech-to-speech model on Amazon Bedrock doing the listening and talking. The model runs in the cloud; the and tools run on the Pi. The write-up was prompted by Strands' bidirectional streaming becoming generally available.

Two problems the write-up describes would show up in other voice apps too. A speaker next to a microphone lets the hear its own voice, mistake it for the user and interrupt itself. Turning on Strands' audio processor adds echo cancellation, which uses the sound being played as a reference so it can remove it from what the microphone picks up. Long conversations can also outlast how long a provider keeps one connection open, so Strands renews the connection at a pause between turns and carries the conversation into the new one.

The fish's movement comes from the same audio stream. A small motor driver board between the Pi and the toy's original motors lets software drive them, and the mouth follows the loudness of the reply measured as it plays, not as it arrives, because audio can sit in a buffer before anyone hears it. The builder says this was their first hardware project and that Fable 5 helped identify wires from photos and wrote the original body-control code.

Terms in this piece · Glossary
  • streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.
  • agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
  • agentic loop — The cycle an agent runs in: decide, call a tool, read the result, decide again — repeating until the goal is met or a stop condition fires.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Recommended reads
Comments

Checking sign-in…

Loading comments…