Garry Tan

A short essay arguing that productive AI coding agents come from a lightweight harness plus composable skills. Explains the context-management and workflow ideas behind the '10-100x' claim. An opinion piece, not a harness you can run.

LangChain has raised $160 million. Three years of development. A billion-dollar valuation. LangSmith, their testing platform, is genuinely sophisticated: trajectory evals, trace-to-dataset pipelines, LLM-as-judge, regression suites, unit test frameworks for tools. They have the pieces. Credit where it's due. But pieces aren't a practice. LangChain gives you testing tools. It never tells you what to test, in what order, or when you're done. There's no opinionated workflow that says, in order: this failure happened now write a skill now write the deterministic code now write unit tests now write LLM evals now add a resolver trigger now eval the resolver now audit for duplicates now smoke test now file correctly That loop doesn't exist. You have to invent it yourself from scattered primitives. $160 million in funding, and most LangChain users still don't test their agents, because the framework gave them a gym membership without a workout plan. Most AI agent "reliability" is vibes-based. Prompt tweaks. Bigger system messages. "Please don't hallucinate" incantations. That stuff decays the moment the conversation gets complex. The frameworks that raised hundreds of millions of dollars to solve this gave you monitoring dashboards and unit test helpers and said "good luck." My agent screwed up twice this week. Neither failure can happen again. Not because I asked nicely. Because I turned each failure into a permanent structural fix: a skill with tests that run every day, forever. What hundreds of millions of dollars of VC capital couldn't buy you, I am going to give it to you today for free in open source. I call the practice "skillify." Once you use it, your agents won't keep making the same mistakes. Here's how it works. Failure 1: The Trip That Was Already in the Database I asked my OpenClaw about an old business trip, nearly ten years back, buried somewhere in calendar history. Simple question. Should take one second. Instead the agent did this: Called the live calend

In "Thin Harness, Fat Skills", I introduced five definitions for building agent systems that actually work. Skills got all the attention. People bookmarked the skill-as-method-call pattern, the diarization concept, the thin harness architecture. Good. Those matter. But the one that got almost no attention is the one that matters most. Resolvers. And the reason they got ignored is the same reason they're so important: they're invisible when they work, and catastrophic when they don't. A resolver is a routing table for context. When task type X appears, load document Y first. That's it. One sentence. But that one sentence is the difference between an agent that compounds intelligence and an agent that slowly forgets what it knows. This is the story of how I learned that the hard way. The 20,000-line confession My CLAUDE.md was 20,000 lines. I'm not proud of this. Every quirk, every pattern, every lesson I'd ever encountered with Claude Code, every convention for my codebase, every edge case I'd been burned by. I kept adding. The file kept growing. It felt productive. It felt like I was making the model smarter. I wasn't. I was drowning it. The model's attention degraded. Responses got slower and less precise. Claude Code literally told me to cut it back. That's when you know you've gone too far — the AI is telling you to stop talking. The instinct is natural. You want the model to know everything. So you cram everything into the system prompt, the instructions file, the context window. You're trying to make the model omniscient by proximity. It doesn't work. You can't make someone smarter by shouting louder. You make them smarter by giving them the right book at the right moment. The fix was about 200 lines. A numbered decision tree. Pointers to documents. When the model needs to file something, it walks the tree: Is it a person? → /people/ directory A company? → /companies/ directory A policy analysis? → /civic/ directory Twenty thousand lines of knowledge, accessib

Sorry, not that kind of naked model. But the disappointment you're feeling right now? That's exactly how Kyle Kingsbury feels about LLMs. Kyle Kingsbury is one of the best systems engineers alive. His Jepsen project spent a decade methodically proving that distributed databases didn't work as advertised. That CockroachDB, MongoDB, Redis, and dozens of others made consistency guarantees they couldn't keep. He published the results, the vendors fixed the bugs, and the entire industry got more honest. Jepsen is a masterwork of applied skepticism. Last week he published a 32-page essay called "The Future of Everything is Lies, I Guess: Bullshit About Bullshit Machines." It's beautifully written, deeply researched, genuinely funny, and wrong about the most important thing. His observations are correct. His conclusion is not. A note on scope: this essay addresses Kingsbury's technical claims: that LLMs are unreliable bullshit machines incapable of producing trustworthy output. His broader concerns about labor displacement, information ecology, and cultural impact are real, separate questions that deserve their own essays. I'm not dismissing them by not addressing them here. I'm addressing the architecture question: does model unreliability make useful systems impossible, or does it make them an engineering problem? I think it's the latter. Kingsbury's essay assumes the former. That's where we disagree. Testing the engine on a bench Kingsbury's essay is structured as a catalogue of LLM failures. He asked Gemini to apply materials to a 3D bathroom rendering. It forgot the toilet and changed the room's shape. He asked Claude to do image-to-image transformation. It produced thousands of lines of JavaScript creating an incomprehensible garble of nonsense polygons. He asked ChatGPT to put white patches on a blue shirt. It changed the color, moved the patches, deleted them. He watched a colleague's LLM claim to download stock data and produce a graph of randomly generated number