Substituting LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → predictions for user responses forfeits the randomization guarantee; identification then rests on unverifiable surrogacy assumptions that weaken as a treatment departs from past experiments. Useful before trusting any synthetic-user evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
articleAI Changed How Spotify Builds. What We Learned (and Fixed) About Quality at Higher Velocity
articleCoding Is No Longer the Constraint: Scaling Developer Experience to Teams and Agents at Spotify
articleIndexing the Data Lake for Online Point Queries
