Does Fine-Tuning Undo Activation Steering? Behavioural Recovery Without Weight-Edit Reversal
Source
Philipp E. Glass, Allan Tucker, Yongmin Li, Alina Miron
Author
Philipp E. Glass, Allan Tucker, Yongmin Li, Alina Miron
Date
Terms in this piece · Glossary
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
alignment — The work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
Steering embedded into weights does not survive routine post-deployment fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → behaviourally, even though the edit remains in the weights — so pre-release weight-level alignmentThe work of making AI systems actually pursue what their builders and users intend, rather than something subtly or dangerously different.Full definition → cannot be assumed to hold downstream.
articleRole Steering of Language Models for Social SimulationsIsaac Song, Mohammed Rehan Parwani, Glenn Matlin, Emile Anand, Akhil Theerthala, Arjun Chatterjee, Maria Kostylew, Yonadav G. Shavit, Sebastien Krier, Mark Riedl
articleSteering Instruction Hierarchies at Inference TimeSiqi Zeng, Sewoong Lee, Han Zhao, Julia Hockenmaier
articleSelf-Evolving Embodied Agents via Skill-Harness EvolutionPeidong Wang, Zhiming Ma, Ying Chang, Xufang Luo, Xiaocui Yang, Shi Feng, Yuqing Yang, Dongsheng Li