Ang Lv (吕 昂)

Ang Lv (吕 昂)

Renmin University of China

I am currently a 3rd year Ph.D. student at Gaoling School of Artificial Intelligence (GSAI) in Renmin University of China, supervised by Prof. Rui Yan. I am dedicated to creating more powerful foundation language models.

News Within a Year

  • May 2025: Autonomy-of-Experts Models is accepted by ICML 2025.

  • Nov 2024: I am awarded the 2024 CIE-Tencent Doctoral Student Research Incentive Program (HunYuan Large Language Model Special Project).

  • Sept 2024 - Jan 2025: 6 accepted papers at EMNLP, NeurIPS, NAACL, IJCAI, and WWW 2025 oral.

  • May 2024: I am awarded the 2024 CCF-Tencent Rhino-Bird Elite Talent Program, mentored by Ruobing Xie.

Publications (First Author)

  • Autonomy-of-Experts Models. Ang Lv, Ruobing Xie, Yining Qian, Songhao Wu, Xingwu Sun, Zhanhui Kang, Di Wang, Rui Yan. Proceedings of the 42nd International Conference on Machine Learning (ICML’25) Link

  • PEAR: Position-Embedding-Agnostic Attention Re-weighting Enhances Retrieval-Augmented Generation with Zero Inference Overhead. Ang Lv*, Tao Tan*, Yining Qian*, Hongzhan Lin, Songhao Wu, Yongbo Wang, Feng Wang, Jingtong Wu, Xin Lu, Rui Yan. The Web Conference (WWW’25 oral) Link

  • Language Models “Grok” to Copy. Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Rui Yan. Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL’25 short). Link

  • An Analysis and Mitigation of the Reversal Curse. Ang Lv*, Kaiyi Zhang*, Shufang Xie, Quan Tu, Yuhan Chen, Ji-Rong Wen, Rui Yan. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP’24). Link

  • Mixture-of-Modules: Reinventing Transformers as Dynamic Assemblies of Modules. Ang Lv*, Zhuocheng Gong*, Jian Guan, Junxi Yan, Wei Wu, Huishuai Zhang, Minlie Huang, Dongyan Zhao, Rui Yan. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP’24). Link

  • Mixture of In-Context Experts Enhance LLMs’ Long Context Awareness. Ang Lv*, Hongzhan Lin*, Yuhan Chen*, Chen Zhu, Yang Song, Hengshu Zhu, Rui Yan. Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS’ 24).

  • Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use. Ang Lv*, Yuhan Chen*, Ting-En Lin, Changyu Chen, Yuchuan Wu, Fei Huang, Yongbin Li, Rui Yan. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL’24). Link

  • Batch-ICL: Effective, Efficient, and Order-Agnostic In-Context Learning. Ang Lv*, Kaiyi Zhang*, Yuhan Chen, Hansen Ha, Tao Xu, Rui Yan. Findings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL ‘24 Findings). Link

  • GETMusic: Generating Music Tracks with a Unified Representation and Diffusion Framework. Ang Lv, Xu Tan, Peiling Lu, Wei Ye, Shikun Zhang, Jiang Bian, Rui Yan, The 34th International Joint Conference on Artificial Intelligence (IJCAI’25).

  • Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation. Ang Lv, Xu Tan, Tao Qin, Tie-Yan Liu, Rui Yan. The 33th International Joint Conference on Artificial Intelligence (IJCAI’24). Link

  • DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations. Ang Lv*, Jinpeng Li*, Yuhan Chen, Gao Xing, Ji Zhang, Rui Yan. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23). Link

  • Envisioning Future from the Past: Hierarchical Duality Learning for Multi-Turn Dialogue Generation. Ang Lv*, Jinpeng Li*, Shufang Xie, Rui Yan. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL’23 oral). Link

  • Target-Side Input Augmentation for Sequence to Sequence Generation. Ang Lv*, Shufang Xie*, Yingce Xia, Lijun Wu, Tao Qin, Tie-Yan Liu, Rui Yan. The 10th International Conference on Learning Representations (ICLR’22). Link

Preprint Papers

  • More Expressive Attention with Negative Weights. Ang Lv, Ruobing Xie, Shuaipeng Li, Jiayi Liao, Xingwu Sun, Zhanhui Kang, Di Wang, Rui Yan. Arxiv, Link

  • Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models. Ang Lv, Yuhan Chen, Kaiyi Zhang, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, Rui Yan. Arxiv. Link

  • PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration. Songhao Wu*, Ang Lv*, Xiao Feng, Yufei Zhang, Xun Zhang, Guojun Yin, Wei Lin, Rui Yan. Arxiv link

Honors and Awards

  • CIE-Tencent Doctoral Student Research Incentive Program (HunYuan Large Language Model Special Project), 1 of 17 selected individuals nationwide（中国电子学会-腾讯博士生科研激励计划 混元大模型专项，全国17人）
  • CCF-Tencent Rhino-Bird Elite Talent Program, 2024, 1 of 50 selected individuals nationwide（中国计算机学会-腾讯犀牛鸟精英人才计划，全国50人）
  • Supported by the Outstanding Innovative Talents Cultivation Funded Programs 2023 of Renmin University of China, 2023 (中国人民大学拔尖创新人才)

Academic Services

  • Conference Reviewer: EMNLP & ACL (ARR Area Chair), ICML, ICLR, WWW, KDD
  • Journal Reviewer: ACM TIST

Internships

  • 2022.09 - 2023.03, Alibaba Damo Academy, Hangzhou.
  • 2023.03 - 2023.09, Microsoft Research, Machine Learning Area, mentored by Xu Tan. Our collaborative efforts are dedicated to the Muzic project, which currently boasts 4k stars on GitHub.
  • 2023.09 - 2024.05, Tongyi Lab, Alibaba, Beijing.
  • 2024.05 - now, Tencent, Beijing.