ProxySPEX exploits a 'hierarchy' property, where a higher-order feature interaction being important implies its lower-order subsets are too, to match SPEX's accuracy with about 10x fewer ablations.
On a modified trolley problem where GPT-4o mini answered correctly only 8% of the time, SHAP blamed the word 'trolley' alone, but SPEX found the real driver was a synergy of both 'trolley' instances plus 'pulling' and 'lever'; replacing all four cut the failure rate to near zero.
Applied to a ResNet on CIFAR-10, ProxySPEX distinguished synergistic training-data interactions (different classes jointly explaining a prediction, e.g. sports-car and truck images explaining an automobile prediction) from redundant near-duplicates reinforcing one concept.
Analyzing attentionThe mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.Full definition →-head interactions by depth on an MMLU task, early layers act in a largely linear/independent regime while later layers show heads interacting heavily within the same layer; a ProxySPEX-informed pruning strategy actually improved target-task performance.
Terms in this piece · Glossary
attention — The mechanism that lets a model weigh which earlier words matter for the word it's currently processing — the core operation of a transformer.
Why it matters
It gives interpretability researchers a way to capture interaction effects that standard attribution methods miss as models grow more complex.
articleScaling Inherently Interpretable Language ModelsGuide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen, Isaac Plant, Muawiz Chaudhary, Nathaniel Monson, Saqib Azim, Zhichen Guo, Julius Adebayo
articleWhen Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey ResponsesZihan Chen, Di Zhu, Lei Nico Zheng
videoSemantic Blindness: 500,000 Sensors Confused an LLM - Raahul Singh & Vanč Levstik, PhaidraAI Engineer