Round 1 of a vibe-coder ladder, judged by an LLM that only sees the UI
Source
alviso
Author
alviso
Date
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
It shows a concrete method for judging vibe-coded apps purely by interacting with the UI (no source access), with the full judge rationale published, an approach worth studying for anyone building AI-output evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → pipelines.
articleVibe Patenting: Evaluating LLM Judges for Professional Patent-Drafting AgentsToshiaki Koike-Akino, Vlad Blaykhman, Ye Wang, Jing Liu, Gene V. Vinokur
articleVibe Coding: Practice, Performance, Productivity, and Risk -A State-of-the-Art ReviewDominik L. Michels, Mutaz Abu Ghazaleh, Francois Lazzari, Nabil Kassem, Jonathan Klein
videoVibe Coding Is The FutureY Combinator