Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google, and Amazon
- Source
- lennysnewsletter.com
- Author
- Lenny Rachitsky
- Date

Context
An AI product can earn greater autonomy through a deliberately limited first version. AI deployment specialists Aishwarya Naresh Reganti and Kiriti Badam describe a customer-support progression: initially, the system suggests replies to human staff, whose corrections reveal where it fails. Once those weaknesses are understood, a team can consider letting it answer customers directly and later perform actions such as refunds. Each stage supplies evidence for the next permission decision. Their proposed workflow makes human correction part of product development, rather than treating full autonomy as the starting requirement.
“You don't know how the user might behave with your product, and you also don't know how the LLM might respond to that.”
Aishwarya Naresh Reganti
“It's not about being the first company to have an agent among your competitors. It's about have you built the right fly wheels in place so that you can improve over time.”
Aishwarya Naresh Reganti
“I probably will go as far to say that if someone's selling you one click agents it's pure marketing, you don't want to buy into that.”
Aishwarya Naresh Reganti
“You must be comfortable with the fact that your intuitions might not be right, and you probably are the dumbest person in the room, and you wanna learn from everyone.”
Aishwarya Naresh Reganti
“talking to an engineer in like any random company, especially outside of the area, you can see like the amount of impact this coding agent can create and the penetration is very low.”
Kiriti Badam
- 00:00Introduction to Aishwarya and Kiriti
- 05:03Challenges in AI product development
- 07:36Key differences between AI and traditional software
- 13:19Building AI products: start small and scale
- 15:23The importance of human control in AI systems
- 22:38Avoiding prompt injection and jailbreaking
- 25:18Patterns for successful AI product development
- 33:20The debate on evals and production monitoring
- 41:27Codex team's approach to evals and customer feedback
- 45:41Continuous calibration, continuous development (CC/CD) framework
- 58:07Emerging patterns and calibration
- 01:01:24Overhyped and under-hyped AI concepts
- 01:05:17The future of AI
- 01:08:41Skills and best practices for building AI products
- 01:14:04Lightning round and final thoughts
podcastThe 100-person AI lab that became Anthropic and Google's secret weapon | Edwin Chen (Surge AI)
podcastWhy humans are AI’s biggest bottleneck (and what’s coming in 2026) | Alexander Embiricos (OpenAI Codex Product Lead)
podcastThe coming AI security crisis (and what to do about it) | Sander Schulhoff
podcastThe non-technical PM’s guide to building with Cursor | Zevi Arnovitz (Meta)
podcastThe rise of the professional vibe coder (a new AI-era job) | Lazar Jovanovic (Professional Vibe Coder)Lenny Rachitsky
podcastAI’s third era: the rise of persistent AI coworkers | Tara Seshan (OpenAI’s product lead)Lenny Rachitsky
podcast“Engineers are becoming sorcerers” | The future of software development with OpenAI’s Sherwin WuLenny Rachitsky
Checking sign-in…
Loading comments…