Imagine if naked people were stupider. It turns out, naked models actually are.
Source
Garry Tan
Author
Garry TanTop Viber
Date
Key takeaways · AI-distilled
The sharpest evidence is Anthropic's own: the leaked Claude Code source is roughly 512,000 lines of scaffolding — repo context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →, prompt cachingReusing the model's processed form of a repeated prompt prefix so subsequent calls skip re-reading it, cutting cost and latency substantially.Full definition →, purpose-built tools, session memory, sub-agents — around a model its makers won't ship bare.
The vocabulary is worth stealing. A agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → file is markdown teaching a procedure, not a prompt. A resolver is a routing table picking which skill to read. Deterministic code handles what must be exact. The agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition → is the thin loop.
The author concedes the cost: 512,000 lines is not simple, harnesses fail, skills encode wrong procedures, and a verification layer is only as good as the invariants it checks. The claim is tractability, not ease.
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
prompt caching — Reusing the model's processed form of a repeated prompt prefix so subsequent calls skip re-reading it, cutting cost and latency substantially.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
Why it matters
Jepsen's Kyle Kingsbury turns his distributed-systems rigor on LLMs, and the results are humbling. Read it for a clear-eyed, adversarial view of where models actually break — an antidote to benchmark hype.
Key quotes
“He's testing the engine on a bench and concluding that cars are unsafe.”
“Because harnesses make the system trustworthy, even when the model inside it is not.”
“That's 512,000 lines of evidence that model reliability is an engineering problem, not a philosophical one.”
“The claim is that they turn model unreliability from a reason-to-stop into a problem-to-solve.”
