Eugene Yan is a member of technical staff at Anthropic, focused on building reliable AI systems for customers. He previously worked on recommendations, retrieval and language-model applications at Amazon and led machine-learning teams at Alibaba/Lazada.[1]
Start with Product Evals in Three Simple Steps →
GitHub 
X Background
Yan identifies his bachelor’s studies as psychology and organizational behavior at Singapore Management University and his master’s as computer science at Georgia Tech. His OMSCS account explains that he pursued structured foundations while already working as a data scientist, completing the program part time. The education record comes from his own corrections to generated biographies, which he published to demonstrate how models can get plausible personal details wrong.[3][4]
Career
His first-person timeline lists IBM data-science work in 2013–2014, data science and machine-learning leadership at Alibaba/Lazada in 2015–2018, a health-technology role in 2018–2019 and Amazon in 2020–2025. At Amazon his work included retrieval, recommendation ranking, summarization, translation and question answering. He joined Anthropic in 2026. These systems address different tasks but share a production concern: connecting model outputs to users’ actual needs and evaluating whether the resulting product works.[1]
Contributions
Evaluating a product instead of a demonstration
His product-evaluation guide starts with representative inputs and outputs labeled by people. He recommends clear pass/fail judgments for objective criteria and win/lose/tie comparisons for subjective ones, then aligning model-based evaluators against those labels. An evaluation harness runs the same checks when system configurations change. The sequence helps teams distinguish an improvement in a specific use case from a model’s general benchmark score or an appealing individual answer.[2]
Explaining retrieval and recommendation systems
His curated writing separates retrieval, which finds candidate items, from ranking, which orders them. It also covers online versus offline system design, real-time recommendations, personalization and language-model application patterns. This supplies engineers with an organizing map of the decisions behind a recommendation or search product, then links to deeper explanations and prototypes.[5]
Tools, writing & talks
Writing, research & talks
Start here
Product Evals in Three Simple Stepseugeneyan.comA practical sequence for human labels, aligned model evaluators and repeated product tests.
Start with product evaluations if you are building an AI feature and need a dependable way to compare changes. Software-development experience helps; the first step is a small labeled dataset, not a complex tool.
Author[2]- Patterns for Building Cybersecurity Evalseugeneyan.com
A sandboxed target, inputs that influence task difficulty, tools, and a grader.
Catalog workOriginal source - Using LLMs to Secure Source Codeeugeneyan.com
Build a threat model, discover vulnerabilities, verify, triage, and patch.
Catalog workOriginal source - How to Work and Compound with AIeugeneyan.com
A working model for compounding output with AI: treat context as infrastructure you maintain, encode taste as configuration, use verification to buy autonomy, scale through delegation, and close the loop so each cycle improves the next.
Catalog workOriginal source An LLM that can converse in English & item IDs, and make recommendations w/o retrieval or tools.
Catalog workOriginal source- Evaluating Long-Context Question & Answer Systemseugeneyan.com
Evaluation metrics, how to build eval datasets, eval methodology, and a review of several benchmarks.
Catalog workOriginal source Applying the scientific method, building via eval-driven development, and monitoring AI output.
Catalog workOriginal sourceA synthesis of how large language models are rewriting recommendation and search systems: architectural shifts in retrieval and ranking, using models to generate training data, changing training paradigms, and the move toward unified frameworks.
Catalog workOriginal sourceLook at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.
Catalog workOriginal sourceUse cases, techniques, alignment, finetuning, and critiques against LLM-evaluators.
Catalog workOriginal sourceSpecial double-feature closing keynote from the 6 authors of the hit O'Reilly article on Applied LLMs.
Catalog workOriginal sourceChallenges and lessons from deploying LLM experiences: evals, scalability, guardrails.
Catalog workOriginal sourceStructured input/output, prefilling, n-shots prompting, chain-of-thought, reducing hallucinations, etc.
Catalog workOriginal source- What We've Learned From A Year of Building with LLMseugeneyan.com
From the tactical nuts & bolts to the operational day-to-day to the long-term business strategy.
Catalog workOriginal source - Task-Specific LLM Evals that Do & Don't Workeugeneyan.com
Evals for classification, summarization, translation, copyright regurgitation, and toxicity.
Catalog workOriginal source - Don't Mock Machine Learning Models In Unit Testseugeneyan.com
How unit testing machine learning code differs from typical software practices
Catalog workOriginal source Overcoming the bottleneck of human annotations in instruction-tuning, preference-tuning, and pretraining.
Catalog workOriginal sourceHow to use open-source, permissive-use data and collect less labeled samples for our tasks.
Catalog workOriginal sourceReference, context, and preference-based metrics, self-consistency, and catching hallucinations.
Catalog workOriginal source- Patterns for Building LLM-based Systems & Productseugeneyan.com
Evals, RAG, fine-tuning, caching, guardrails, defensive UX, and collecting user feedback.
Catalog workOriginal source - Some Intuition on Attention and the Transformereugeneyan.com
What's the big deal, intuition on query-key-value vectors, multiple heads, multiple layers, and more.
Catalog workOriginal source - Minimal Chateugeneyan.com
An argument that chat became the default LLM interface by inertia rather than fit, and a case for interaction patterns that expose model capability more directly than a conversational text box.
Catalog workOriginal source - More Design Patterns For Machine Learning Systemseugeneyan.com
9 patterns including HITL, hard mining, reframing, cascade, data flywheel, business rules layer, and more.
Catalog workOriginal source - Georgia Tech OMSCS FAQeugeneyan.com
A first-person account of studying computer science part time while working in industry.
Author[4] - Start Here: applied ML and AI systemseugeneyan.com
A curated guide to his writing on recommendations, retrieval, language models and engineering practice.
Author[5]