I am currently a second year Ph.D. student at Gaoling School of Artificial Intelligence (GSAI) in Renmin University of China, supervised by Prof. Rui Yan. I focus on long-context ability and mechanistic interpretability of LLMs. I am dedicated to creating more powerful foundation models.

News Within a Year

  • 2024.05 I am successfully selected by 2024 CCF-Tencent Rhino-Bird Elite Talent Program, mentored by Dr. Ruobing Xie.

  • 2024.05 Two papers are accepted by ACL 2024 main conference. One paper is accepted by ACL 2024 findings.

  • 2024.04 One paper is accepted by IJCAI 2024.

  • 2024.03: We thoroughly studied the mechanisms of factual recall in Transformer-based language models, and I hope you will find the exciting findings engaging!

  • 2023.11: We found that using the next-token prediction training paradigm leads to insufficient training data utility, and the model cannot fully grasp knowledge in the training data, leading to the “reversal curse.”

Publications (First Author and First Co-author)

  • Yuhan Chen*, Ang Lv*, Ting-En Lin, Changyu Chen, Yuchuan Wu, Fei Huang, Yongbin Li, Rui Yan. Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL’24). Link

  • Kaiyi Zhang*, Ang Lv*, Yuhan Chen, Hansen Ha, Tao Xu, Rui Yan. Batch-ICL: Effective, Efficient, and Order-Agnostic In-Context Learning. Findings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL’24). Link

  • Ang Lv, Xu Tan, Tao Qin, Tie-Yan Liu, Rui Yan. Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation, The 33th International Joint Conference on Artificial Intelligence (IJCAI’24). Link

  • Ang Lv, Jinpeng Li, Yuhan Chen, Gao Xing, Ji Zhang, Rui Yan. DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23). Link

  • Ang Lv, Jinpeng Li, Shufang Xie, Rui Yan. Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23 oral). Link

  • Shufang Xie*, Ang Lv*, Yingce Xia, Lijun Wu, Tao Qin, Tie-Yan Liu, Rui Yan. Target-Side Input Augmentation for Sequence to Sequence Generation. The 10th International Conference on Learning Representations (ICLR’22). Link

Other Publications

  • Changyu Chen, Xiting Wang, Ting-En Lin, Ang Lv, Yuchuan Wu, Xin Gao, Ji-Rong Wen, Rui Yan, Yongbin Li. Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL’24). Link

Preprint Papers

  • Ang Lv, Yuhan Chen, Kaiyi Zhang, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, Rui Yan. Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models, Arxiv. Link

  • Ang Lv, Kaiyi Zhang, Shufang Xie, Quan Tu, Yuhan Chen, Ji-Rong Wen, Rui Yan. Are We Falling in a Middle-Intelligence Trap? An Analysis and Mitigation of the Reversal Curse, Arxiv. Link

  • Ang Lv, Xu Tan, Peiling Lu, Wei Ye, Shikun Zhang, Jiang Bian, Rui Yan. GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework, Arxiv. Link

  • Hongzhan Lin*, Ang Lv*, Yuhan Chen, Chen Zhu, Yang Song, Hengshu Zhu, Rui Yan. Mixture of In-Context Experts Enhance LLMs’ Long Context Awareness, Arxiv. Link

  • Zhuocheng Gong*, Ang Lv*, Jian Guan, Junxi Yan, Wei Wu, Huishuai Zhang, Minlie Huang, Dongyan Zhao, Rui Yan. Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules, Arxiv. Link

Honors and Awards

  • Supported by the Outstanding Innovative Talents Cultivation Funded Programs 2023 of Renmin University of China, 2023 (中国人民大学拔尖创新人才)
  • CCF-Tencent Rhino-Bird Elite Talent Program, 2024(CCF-腾讯犀牛鸟精英人才计划)

Internships

  • 2022.09 - 2023.03, Alibaba Damo Academy, Hangzhou.
  • 2023.03 - 2023.09, Microsoft Research, Machine Learning Area, mentored by Xu Tan. Our collaborative efforts are dedicated to the Muzic project, which currently boasts 4k stars on GitHub.
  • 2023.09 - 2024.05, Alibaba Damo Academy, Beijing.
  • 2024.05 - now, Tencent, Beijing.