Interpretability: Understanding how AI models think
Source
youtube.com
Author
Anthropic
Date
Context
Changing an internal representation can help researchers test how a model reached an answer. In Anthropic'sinterpretability discussion, researchers describe asking for the capital of the state containing Dallas. They identified an intermediate representation associated with Texas; replacing it with California caused the model to answer Sacramento.
That intervention provides stronger evidence for an intermediate state-selection step than the original correct answer alone. It illustrates what interpretability research tries to recover: a testable account of the computation connecting a question to its answer.