The Self-Improving OSS Agent Stack — Marc Klingen, Langfuse

Describes a reference stack linking production traces to datasets and evals, and shows where agents can take over the improvement loop. Teams can apply it to cut manual trace review.

• eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
• AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.

• videoHarness Engineering: How to Build a Software Factory — Dru Knox, Tessl
• videoWhy Building an Eval Platform Is Harder Than It Looks — Braintrust
• videoMove Fast and Don't Break Things: Scaling Databases for the AI Era — PlanetScale
• videoCooking with Codex — Charlie Guo & Gabriel Chua, OpenAI

• videoImproving Agents is a Data Mining Problem — Vivek Trivedy, LangChainAI Engineer
• videoSkill issue: Lessons from skilling up coding agents to use Langfuse - Marc Klingen, ClickhouseAI Engineer
• videoFrom Vibes to Production: Evaluating and Shipping AI Agents That Work 201 — Laurie Voss, Arize AIAI Engineer

Checking sign-in…

Loading comments…