Distill to Detect: Amplifying Hidden Bias in Fine-Tuned LLMs
- Source
- x.com
- Date
How do you find a hidden bias in a fine-tuned LLM that you don’t know to look for? Amplify it. Distill to Detect (D2D) surfaces hidden biases by distilling the shift between a suspected model and its base into a small cartridge. This amplifies stealth preferences into generated text, making subliminal signals visible to existing auditing methods. Congrats to @talaei_shayan, @AbhinavChinta10, @Devvrit_Khatri, @aminkarbasi, @Azaliamirh, and Amin Saberi!
Context
Stanford AI Lab highlights a method called Distill to Detect (D2D) for finding a hidden bias in a fine-tuned when nobody knows what to look for. In its post, the lab says D2D distills the shift between a suspected model and its base model into a small cartridge, which amplifies stealth preferences into generated text so that existing auditing methods can see them.
The paper reports one concrete result: an existing bias-auditing tool detected a hidden preference for the brand Fanta in 33% of cases when applied to the fine-tuned model directly, and in 100% of cases after the shift was into a cartridge. That is a reported result for one induced bias, not evidence that every hidden bias becomes detectable this way.
- LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
- distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
articleChain-of-Models: Cross-Model Auditing for Bias-Robust LLM JudgesQian Wang, Zhanzhi Lou, Zhenheng Tang, Nuo Chen, Bingsheng He
articleFairLint-DL: An IDE-Native Tool for Fairness Debugging of Deep Learning SoftwareArchit Rathod, Saeid Tizpaz-Niari
postDistillKit ships logit compression and pre-captured teacher dataArcee.ai
Checking sign-in…
Loading comments…