distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
Why it matters
A dense, practitioner-written distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → of what actually works when shipping LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → apps — covering prompting patterns, RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition →, evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → strategy, and the operational/organizational realities most tutorials skip.
Key quotes
“Nonetheless, while the barrier to entry for building with AI has been lowered, creating products and systems that are effective—beyond a demo—remains deceptively difficult.”
Eugene Yan
“If n is too low, the model may over-anchor on those specific examples, hurting its ability to generalize. As a rule of thumb, aim for n ≥ 5. Don’t be afraid to go as high as a few dozen.”
Eugene Yan
“By switching from a single prompt to a multi-step workflow, they increased GPT-4 accuracy (pass@5) on CodeContests from 19% to 44%.”
Eugene Yan
“Thus, the likelihood that an agent completes a multi-step task successfully decreases exponentially as the number of steps increases.”
Eugene Yan
“The most successful agent builders may be those with strong experience managing junior engineers because the process of generating plans is similar to how we instruct and manage juniors.”
