Ang Lv (吕昂) is currently a second year Ph.D. student at Gaoling School of Artificial Intelligence (GSAI) in Renmin University of China (RUC), supervised by Prof. Rui Yan.
My research primarily focuses on NLP generation tasks. Starting in 2023, I shifted my focus towards mechanistic interpretability. Through a deep understanding of language models, I am dedicated to creating more powerful foundation models.
🔥 News Within a Year
-
2024.03: I thoroughly studied the mechanisms of factual recall in Transformer-based language models, and I hope you will find the exciting findings engaging!
-
2023.12: I found some drawbacks in RoPE, which limit the LLMs’ context awareness to some specific positions. My coauthor and I mitigated this issue and enhanced a 7B model to outperform GPT-4 in a tool-use benchmark!
-
2023.11: I found that using the next-token prediction training paradigm leads to insufficient training data utility, and the model cannot fully grasp knowledge in the training data, leading to the “reversal curse.”
📝 Publications
DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations
Ang Lv, Jinpeng Li, Yuhan Chen, Gao Xing, Ji Zhang, Rui Yan
Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation
Ang Lv, Jinpeng Li, Shufang Xie, Rui Yan
Target-Side Input Augmentation for Sequence to Sequence Generation
Shufang Xie*, Ang Lv*, Yingce Xia, Lijun Wu, Tao Qin, Tie-Yan Liu, Rui Yan
Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
Ang Lv, Xu Tan, Tao Qin, Tie-Yan Liu, Rui Yan
🎖 Honors and Awards
- 2023.09 Doctoral Scholarship for Elite Innovative Talents of Renmin University of China (中国人民大学拔尖创新人才)
💻 Internships
- 2022.09 - 2023.03, Alibaba Damo Academy, Hangzhou, China.
- 2023.03 - 2023.09, Microsoft Research, Machine Learning Area, mentored by Xu Tan. Our collaborative efforts are dedicated to the Muzic project, which currently boasts 4k stars on GitHub.
- 2023.09 - 2023.03, Alibaba Damo Academy, Beijing, China.