In this pyramid is how I think personally think about modern AI evaluation systems. At the bottom, you can see they are benchmarks. They are useful. They are scalable. They are repeatable. But their operational value is limited. In the middle, there are scenario-based evaluations. These simulate realistic workflows. And at the very top, you see production telemetry. This is where the highest value evaluation signals come from. The surprising insight is that the most evaluation data often comes from real users interacting with real systems. Now, let's talk about AI offline evaluations. So, offline evaluation still matters, but the methodology changes. Instead of evaluating prompts, we evaluate scenarios. For example, a customer support workflow, a code generation workflow, a research workflow. The agent operates inside that simulated environment. We measure the task completion rate, tool correctness, planning quality, resource usage, which is which becomes exponentially high at high scale. The key takeaway, agent evaluation should be scenario driven, not prompt driven. Once a system reaches production, every interaction becomes a signal. This is one of the biggest shifts in evaluation thinking. Production traffic is no longer just traffic. It becomes evaluation data. We collect execution traces, user outcomes, escalations, failures, feedback signals. Production is the largest and the most representative evaluation data any organization will ever have. Many organizations view humans as fallback systems. I think that's a wrong framing. Humans are the evaluators. They provide signals that automated systems cannot. They assess correctness, trust, usefulness, safety. These signals become really critical for calibrating evaluation pipelines and identifying blind spots in automated metrics. The most successful systems combine automated evaluation with targeted human review.