Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse

Shows how to turn domain-expert feedback into evaluators and block prompt changes in CI when scores drop, for applications where errors are costly.

• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.

• videoAn AI Agent Got Its User Banned From Resy — Keycard, JPMorgan Chase & AAuth
• videoTeach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
• videoHow to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
• videoInsurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs

• videoContinuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouseAI Engineer
• videoHow to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AIAI Engineer
• videoFrom Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AIAI Engineer

Checking sign-in…

Loading comments…