← All IntelClip / AI ToolsOffline dictation app built from two fine-tuned tiny Gemma models
From Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google · ≈17:48
A shipped production architecture — ASR engine plus a text-policing engine that strips filler words and biases toward personal names — that replaces a subscription cloud feature entirely on-device.
What’s in it
- A shipped production architecture — ASR engine plus a text-policing engine that strips filler words and biases toward personal names — that replaces a subscription cloud feature entirely on-device.
Clip transcript
deploying at like very wide scale. So here's another example in this is one example in production where we have this is an app that we've developed for voice dictation without subscription. Um all of the voice dictation happens locally on device. Um, and as well as just doing dictation, it also does uh it also does um wow, it kind of cleans up ums and a's, right? If you see on the right hand side, it's able to clean up text. It's also able to do biasing towards kind of words and names that um uh are kind of relevant to you personally. So kind of personalization. The left hand side kind of shows how we built that application. So there's an ASR engine and a text policing engine. And both of these are fine-tuned versions of tiny Gemma models. And this allows us to take something that would have been a kind of like server only feature of, you know, where you require a subscription to do highly accurate uh voice dictation and have an app that's just able to do that completely offline with very very good quality. Uh so this is something you can try on iOS if if you want to give this a go today. But um uh yeah and it just uh the backbone of this app is kind of two fine-tuned uh small gemma based models in the low singledigit hundreds of parameter million parameters. We also worth uh noting is there's also uh kind of features in developer preview in Chrome for example that kind of summarization and proofre APIs or feature as built-in APIs in Chrome and delivering those features via tiny models allows um the Chrome team to ship them to a much wider set of uh Chrome users than would otherwise be possible. Um yeah, so that's uh we've probably g
Comments
Checking sign-in…
Loading comments…