“Thus, at its core, prompt engineering is about conditioning the probabilistic model to generate our desired output. Thus, each additional instruction or piece of context can be viewed as conditioning that steers the model’s generation in a particular direction.”

Eugene Yan

“I typically have at least a dozen samples or more. Most academic evals use 32-shot or 64-shot prompts.”

Eugene Yan

“By guiding the model to look for specific information and verify its intermediate outputs against the source document, we can significantly improve factual consistency (i.e., reduce hallucination). In some cases, we’ve observed that adding a sentence or two to the CoT prompt removed the majority of hallucinations.”

Eugene Yan

“As an example, AlphaCodium shared that by switching from a single direct prompt to a multi-step workflow, they increased gpt-4 accuracy (pass@5) on CodeContests from 19% to 44%.”

Eugene Yan

“My rule of thumb is to start with a temperature of 0.8 and then lower it as necessary. What we want is the highest temperature that still leads to good results for the specific task.”

Eugene Yan