Intel / article

Round 1 of a vibe-coder ladder, judged by an LLM that only sees the UI

Source: vibeladder.dev
Author: alviso
Date: Published Sep 16, 2026

Context

Vibeladder's first round evaluated the applications people built with AI through their interfaces. The September 2026 results describe two submitted notes apps, both of which passed the five-item functional checklist. An then compared them by operating their interfaces without reading their source code. The reported difference emerged in richer Markdown support, layout behavior with long notes and additional features.

The organizer also corrected its test after discovering it had dismissed a confirmation dialog and incorrectly marked one app's delete button as broken. That correction makes the itself part of the lesson: a failed automated check can reflect the tester's behavior. With two entries, different tools and different amounts of iteration, this round illustrates a testing approach; it cannot isolate which model is better.

Terms in this piece · Glossary

LLM-as-judge — Using one model to score another's output against a rubric, so quality can be measured at a scale human grading cannot reach.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Read the source vibeladder.dev