Library for building synthetic agent environments [MIT]
Source
kiln.tech
Author
scosman
Date
Why it matters
AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → need a known start state, stateful tools, and state diffs. Seahaven provides isolated per-run worlds so you can grade the change an agent made instead of its transcript, without touching production.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
videoAgents Without Code: Skills, YAML, and Filesystems Replaced Python — Philipp Schmid, Google DeepMindAI Engineer
videoEverything Is a Rollout — Alex Shaw + Ryan Marten, Terminal-Bench, Harbor, Laude InstituteAI Engineer