Learning To Replicate Expert Judgment In Financial Tasks

Bridgewater AIA Labs, a team at the investment firm Bridgewater, reports that a model it fine-tuned agreed with its investors' document labels more often than the general-purpose models it tested, even after those models were given expert-written prompts. The team trained on Tinker, a training tool from Thinking Machines, and cleaned its training labels by sending examples where a trained model disagreed with non-expert labels to experts for re-evaluation. It then scored the model on held-out test data. Across six document-triage tasks, such as judging whether an article is relevant to an investor or finding where boilerplate begins, the team reports average accuracy of 84.7%, against 78.2% for the best general-purpose model it evaluated. In this research report, accuracy means agreement with the investors' own labels, so the result concerns document triage and does not measure investment returns.