The Goldilocks Of Pragmatic Understanding Fine Tuning Strategy Matters For Implicature Resolution By Llms 2022 10 26

Understanding an indirect answer requires connecting the words to the situation. A response about wearing gloves, for example, can imply that someone left no fingerprints. A 2022 study abstract, hosted by Cohere, describes testing whether language models could resolve such implied meanings into yes-or-no answers. Three of four model categories performed near chance, while models instruction-tuned at the example level performed substantially better. The result identifies a specific conversational ability that depended on fine-tuning strategy in those experiments.