context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
A rigorous, practitioner-oriented survey of in-context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → prompting methods — few-shot, chain-of-thoughtHaving a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.Full definition →, self-consistency, and more — that explains not just what works but the empirical reasoning behind steering LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → behavior without fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition →.
Key quotes
“It is an empirical science and the effect of prompt engineering methods can vary a lot among models, thus requiring heavy experimentation and heuristics.”
“At its core, the goal of prompt engineering is about alignment and model steerability.”
articlePromptimus: Improving already good LLM prompts with zero manual engineeringwww.amazon.science
articleBeyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on QualityQipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu
repopromptbaseMicrosoft