Anthropic's 2024 election testing addressed a problem beyond deliberate disinformation: people asking ordinary voting questions could receive outdated answers. External experts identified examples, and the company used those examples to generate larger test sets. It then changed model instructions to disclose knowledge limits and adjusted training to encourage referrals to authoritative sources.
The June 2024 account reports improvements in those response behaviors. The mechanism is to help users recognize when they need a current source. More frequent referrals and disclosures are useful safeguards, but measuring them does not establish that every voting answer became accurate.
Terms in this piece · Glossary
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
