Intel / article

Customer Experience (CX) Agents in Production: Lessons from Lyft, Vodafone, and LATAM Airlines

Source: langchain.com
Author: LangChain
Date: Published Aug 4, 2026

Context

A support can look successful in testing because its simulated customers are unusually cooperative. In LangChain's production case study, ride-hailing company Lyft initially saw simulated-conversation pass rates above 90%, but the artificial customers were more articulate and patient than real users.

The team revised its simulated customers using actual customer wording and distinct personas, then checked automated judges against human reviewers. It also replaced broad quality scores with specific pass-or-fail checks. This made failures more actionable: a team could identify incorrect information or mishandled escalation and change the relevant behavior.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

Key quotes

“LATAM Airlines saw this with Concierge. Initially, 13% of messages were classified as out of scope. After reviewing the conversations, the team found that 95% were legitimate passenger needs the agent had not yet been designed to handle, including check-in and baggage questions. Adding a customer-care specialist reduced the out-of-scope rate from 13% to 1%.”
LangChain

“Lyft facilitates 79 million trips each month, while AI Assist handles roughly 270,000 monthly interactions across seven or more production agents. The system has achieved a 65% deflection rate and a 35% AI resolution rate.”
LangChain

“This approach reduced the time required to develop an agent from roughly six months for Lyft’s first driver agent to about two weeks for a new configurable agent.”
LangChain

“Lyft’s first LLM-generated customers were too articulate, patient, and cooperative, producing offline pass rates above 90% that did not reflect production behavior. Real users often write in fragments, omit context, repeat themselves, or arrive with a specific goal such as securing a refund or bypassing the agent.”
LangChain

“Super TOBi is the agentic evolution of Fastweb + Vodafone’s existing chatbot. It now serves nearly 9.5 million customers across the Customer Companion App and voice channels, handling use cases such as cost control, active offers, roaming, sales, and billing.”
LangChain

Read the source www.langchain.com