Skip to content
CaMLSysUniversity of Cambridge
Home People Publications Projects Contact

← All publications

LLM Unlearning via Neural Activation Redirection

  • William ShenUniversity of Cambridge
  • Xinchi QiuUniversity of CambridgeMeta
  • Meghdad KurmanjiUniversity of Cambridge
  • Alexandru-Andrei IacobUniversity of Cambridge
  • Lorenzo SaniUniversity of Cambridge
  • Yihong ChenUniversity College London
  • Nicola CanceddaMeta
  • Nicholas D. LaneUniversity of Cambridge

Conference on Neural Information Processing Systems (NeurIPS), 2025

Abstract

The ability to selectively remove knowledge from LLMs is highly desirable. However, existing methods often struggle with balancing unlearning efficacy and retain model utility, and lack controllability at inference time to emulate base model behavior as if it had never seen the unlearned data. In this paper, we propose LUNAR, a novel unlearning method grounded in the Linear Representation Hypothesis and operates by redirecting the representations of unlearned data to activation regions that expresses its inability to answer. We show that contrastive features are not a prerequisite for effective activation redirection, and LUNAR achieves state-of-the-art unlearning performance and superior controllability. Specifically, LUNAR achieves between 2.9x and 11.7x improvement in the combined unlearning efficacy and model utility score (Deviation Score) across various base models and generates coherent, contextually appropriate responses post-unlearning. Moreover, LUNAR effectively reduces parameter updates to a single down-projection matrix, a novel design that significantly enhances efficiency by 20x and robustness. Finally, we demonstrate that LUNAR is robust to white-box adversarial attacks and versatile in real-world scenarios, including handling sequential unlearning requests.

Materials

  • PDF
  • Code
BibTeX
@inproceedings{shen2025lunar,
  title = {{LLM Unlearning via Neural Activation Redirection}},
  author = {Shen, William and Qiu, Xinchi and Kurmanji, Meghdad and Iacob, Alexandru-Andrei and Sani, Lorenzo and Chen, Yihong and Cancedda, Nicola and Lane, Nicholas D.},
  year = {2025},
  booktitle = {Conference on Neural Information Processing Systems (NeurIPS)},
  volume = {38},
  pages = {44253--44290},
  publisher = {Neural Information Processing Systems Foundation, Inc. (NeurIPS)},
  doi = {10.52202/085713-1475},
  url = {https://proceedings.neurips.cc/paper_files/paper/2025/hash/3ed6a903be20e1478aa6793f14938091-Abstract-Conference.html}
}
Cambridge Machine Learning Systems Group
Department of Computer Science and Technology
University of Cambridge
ContactGitHub