I am a PhD student at University of Manchester, supervised by Prof. Sophia Ananiadou. Previously, I worked as an NLP researcher at Tencent Technology, where I designed deep learning models for code detection and code matching. I received my Bachelor’s and Master’s degrees from Shanghai Jiao Tong University, supervised by Prof. Gongshen Liu.

My research lies in understanding the inner mechanisms of LLMs and multimodal LLMs. I believe that deeper insights into these models can inform the design of more robust, controllable, and efficient architectures. My current work focuses on two main directions:

a) Mechanistic interpretability of LLMs and multimodal LLMs. I develop and apply interpretability techniques to investigate how LLMs powerfully perform different tasks and capabilities within one single model. My studies span fundamental abilities—such as factual knowledge, arithmetic, and in-context learning—as well as higher-order capabilities like latent multi-hop reasoning and visual question answering. Through these efforts, I aim to establish principled insights that inform and guide the future development of LLMs.

b) Mitigating the catastrophic forgetting problem in LLM post-training. While scaling data and compute has led to significant improvements in the performance of LLMs, it does not address certain fundamental challenges at the mechanistic level. One such challenge is catastrophic forgetting—the tendency of LLMs to lose previously learned capabilities during SFT and RL. My current research aims to investigate the underlying mechanisms of this phenomenon and to develop effective mitigation strategies, with the broader goal of enabling more robust and continually improving LLMs.

Feel free to contact me at zepingyu@foxmail.com if you’re interested in working together.

🔥 News

2025.5: New preprint: Locate-then-Merge: Neuron-Level Parameter Fusion for Mitigating Catastrophic Forgetting in Multimodal LLMs. This work investigates how to mitigate the catastrophic forgetting problem after visual instruction tuning in multimodal LLMs.
2025.2: New preprint: Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Models. This work investigates the mechanism of latent multi-hop reasoning and propose the back attention module to enhance the latent multi-hop reasoning ability in LLMs.
2025.1: New preprint: Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing. This work investigates the mechanism of gender bias and proposes a neuron-level model editing method to reduce gender bias in LLMs without hurting the existing abilities.
2024.12: I’ve compiled paper lists of SAE and neuron in LLMs.
2024.11: New preprint: Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering. This work explores the mechanism of Llava in visual question answering.
2024.09: Our work is accepted by EMNLP 2024 (main): Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis. This work explores the neuron-level information flow of arithmetic mechanism in LLMs and proposes a model pruning method for arithmetic tasks.
2024.09: Our work is accepted by EMNLP 2024 (main): Neuron-Level Knowledge Attribution in Large Language Models. This work introduces how to identify important neurons in LLMs, and explores the neuron-level information flow of factual knowledge mechanism.
2024.09: Our work is accepted by EMNLP 2024 (main): How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning. This work explores the mechanism of in-context learning in LLMs.
2024.04: I’ve compiled a paper list for those interested in exploring the mechanisms of LLMs.