LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
RAG — Retrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
A concentrated set of production LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → lessons from six practitioners covering evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →, prompting, RAGRetrieval-augmented generation — fetching relevant documents first and pasting them into the model's context so it answers from your data instead of memory.Full definition → vs. fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → tradeoffs, and operational pitfalls—useful if you're moving an LLM feature from demo to reliable product.
Key quotes
“Have you ever attended a talk with six speakers? Me neither. It was challenging to figure out a format that was entertaining, educational, and inspiring for the audience.”
Eugene Yan
“We eventually converged on the paired speaker format (aka 2 x 2 x 2), which on hindsight, I think was the right decision. While it needed more preparation and coordination, it resulted in a closing keynote that was higher signal-to-noise and energy.”
Eugene Yan
“We resolved the debate by writing one-pagers for the panel format and 2 x 2 x 2 format.”
