Mitigating memorization in LLMs

A language model can appear good at predicting past events because its training data already contains the outcomes. A Jane Street summer intern project investigated that problem using cricket match previews, comparing performance before and after a model's training cutoff.

The most promising method, divergence decoding, adjusts a large model's predictions at inference time using two smaller models: one trained on data before 2015 and one trained up to the large model's own knowledge cutoff. Steering away from what the later model adds aims to make the large model behave as if it knew only the earlier period, without retraining it. On the cricket dataset it reduced performance on matches from before the cutoff, where memorization would help, without harming performance after the cutoff, which the project presents as evidence of reduced memorization. The work was exploratory and produced no working de-memorizer, and the pre- and post-cutoff matches differ in kind, which complicates the comparison.