LLM-as-judge — Using one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → design is where most AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → and generative pipelines silently break, and this names specific failure modes.
Key quotes
“the hard part was never how to make video. The hard part was how do we generate um good enough video and how do we judge if the video is good enough?”
Maor Bril
“If you think about what is video, video is a storytelling medium.”
Maor Bril
“But if I show you two videos and I'll ask you which one of them is telling a better story, the grand majority will probably agree that B is telling a better story than A, right?”
Maor Bril
“Now, we shipped V1, and it was so wrong. It It was wrong, but it was wrong in a very confident way.”
Maor Bril
“it scored the vibe as opposed to the the the axes”
