Meta’s Muse: How we launched the most successful consumer product this year | Mona Sarantakos
Source
lennysnewsletter.com
Author
Lenny Rachitsky
Date
Why it matters
Shows how a large lab approached launching a consumer-facing agent: delaying release for safety and trust, and designing a personal character interface. Useful for anyone building agent products for non-technical users.
Key takeaways · AI-distilled
Muse took about seven months to launch, by host Lenny's count. Head of product Mona says most of it went to reliability, safety and security: public benchmarks fell short for an AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → that holds personal context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → and buys things, so the team set its own bar and red-teamed.
Muse's evals are tiered: priority evals such as safety and prompt injectionAn attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.Full definition → must pass for any change, secondary evals may trade off slightly, and teams own feature-level evals. Categories span capability, behavior (follow-ups, intent understanding) and priority use cases.
The team requires Muse's system promptThe standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.Full definition → and behavioral spec to be written entirely by hand, with no AI-written part, and has people check that eval prompt distributions match real product use. Mona says skipping human review of the data has consistently led to issues.
Meta AI chats were mostly single-turn, so the team lacked production data for long multi-turn tasks outside coding. Agent swarms read conversations and feedback and generated synthetic data for eval sets; engineers now swarm suspected issues to confirm whether they are real.
Mona calls activation Muse's top problem. Word of mouth from relatable users beat advertising, an Ideas tab of starter packs targets first-prompt freeze, and onboarding built for early adopters won't fit later cohorts, so the team is moving to tiered onboarding variants.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
prompt injection — An attack that hides instructions in content an AI will read — a webpage, email, or document — tricking it into following the attacker instead of the user.
system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
podcastWhat it actually takes to build frontier models | Tulsee Doshi (Head of Product for Gemini)
podcastInside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)
podcastOpenAI’s Head of ChatGPT: We’re entering a new era of AI (again) | Tibo Sottiaux
podcastHow we built Grok Bot in a month | Roman Ugarte (SpaceXAI)
