Prompting Long Context

In a September 2023 study, Anthropic found that asking its model to pull out relevant quotes before answering helped it recall facts from very long documents. In its prompting case study, researchers built long inputs of roughly 70,000 to 95,000 tokens by stitching together sections of a government publication, then tested multiple-choice questions about those sections, mainly on Claude Instant 1.2. Extracting quotes into a scratchpad helped in every head-to-head comparison. Adding examples of correctly answered questions drawn from the same document helped, and more examples helped more, while two generic general-knowledge examples did not seem to. Position mattered too: gains were substantial for answers at the beginning and middle of the input, but performance on answers at the very end could be degraded. The findings apply to these 2023 models and this constructed task, not to current models.