Yifan Gao

• Senior Applied Scientist, Amazon Rufus LLM — Palo Alto, CA (2023–Present)
  Rufus Post-training, Agent & Evaluation
  • Focused on agentic reinforcement learning for Rufus (global/query-level rubrics, RLVR, agentic search), enabling personalized recommendations, comparisons, and product Q&A. [Amazon News]
  • Led agent evaluation for Rufus post-training: worked cross-functionally to translate product requirements into evaluation SOPs, launched human labeling pipelines, and developed an LLM-as-a-judge spanning rubric-based, preference-based, and verifiable judges.
  • Designed memory for context-aware assistance (static + dynamic memory) and developed personalized evaluation rubrics aligned to user intent, preferences, and communication style.
  • Improved general agentic capability through mid-training for function calling [NAACL 2025] and by initiating MCP-based post-training for diverse tool-use environments.
  Rufus LLM Pre-training
  • Led LLM scaling-law development for Rufus foundational models, studied architecture and training dynamics, built a capability prediction recipe, and guided pre-training data quality, mixture, and curriculum design. [TMLR Paper]
  • Led the Pre-training Evaluation suite; defined base-model capability measurements and shopping-critical benchmarks (e.g., [Shopping MMLU - NeurIPS Paper], [Amazon KDD Cup]) and established evaluation principles and a customer-behavior benchmark suite to guide behavior-focused pre-training.
  • Delivered Rufus base models (ultra-sparse MoE) that match or exceed leading open models (e.g., DeepSeek-V3-Base, Kimi-K2-Base) across key benchmarks (MMLU, MMLU-Pro, Math) and internal shopping benchmarks.
• Applied Scientist, Amazon A9 Search (2022–2023)
  • Led a retrieval-augmented media query parsing model and a million-scale media knowledge graph, driving major gains in title/author recognition and search relevance and delivering multi-million revenue impact. [COSMO KG]
• Applied Scientist Intern, AWS AI Labs (2020 Summer)
  • Collaborators: Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Ramesh Nallapati
  • Proposed a generative evidence fusion approach for ambiguous open-domain question answering, published in ACL 2021, and ranked 1st on the AmbigQA Leaderboard (Oct. 2020).
• Research Intern, Salesforce Research (2019 Fall)
  Collaborators: Chien-Sheng (Jason) Wu, Shafiq Joty, Steven C.H. Hoi
  • Worked on implicative reasoning in conversational machine reading and proposed a new method to explicitly track entailment states in multi-turn interactions, published in ACL 2020, and ranked 1st on the ShARC Leaderboard.
• PhD Student, Chinese University of Hong Kong (2017–2021)
  During my PhD study, I teach machines to ask and answer reading comprehension questions driven by two goals:
  • Knowledge Assessment: Asking questions is an effective approach to assess the understanding of users towards a passage of text. I work on 1) how to ask questions with different levels of difficulty [IJCAI'19]; 2) how to ask interconnected questions in a conversation for interactiveness and persistence [ACL'19], and 3) how to generate distractors (wrong options) in multiple-choice reading comprehension questions [AAAI'19].
  • Information Acquisition: Many questions are inherently underspecified or ambiguous in our day-to-day conversations. In this case, the machine needs to 1) keep asking follow-up questions in a dialog until it gathers enough information to answer those high-level questions [ACL'20] [EMNLP'20] [arXiv'21]; or 2) present a series of disambiguated QA pairs when the query is ambiguous[ACL'21].

• Chinese University of Hong Kong — Hong Kong (2017–2021)
  Ph.D. in Computer Science
  Advisors: Irwin King, Michael R. Lyu; Thesis Committee: Yufei Tao (Chair), Jeffrey Xu Yu, Hsin-Hsi Chen
  Thesis: Teaching Machines to Ask and Answer Questions
• Beihang University — Beijing (2013–2017)
  B.Eng. in Electronic and Information Engineering
• Technische Universität Darmstadt — Darmstadt (2016)
  Exchange Student, Computer Science

   Google Scholar

*: equal contribution; †: intern that I mentored

• [ICML'25] M+: Extending MemoryLLM with Scalable Long-Term Memory
  Yu Wang†, Dmitry Krotov, Yuanzhe Hu, Yifan Gao, Wangchunshu Zhou, Julian McAuley, Dan Gutfreund, Rogerio Feris, Zexue He
  ICML 2025. [arxiv]
• [ACL'25] EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association
  Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song
  ACL 2025. [arxiv]
• [ACL'25] UniConv: Unifying Retrieval and Response Generation for Large Language Model in Conversation
  Fengran Mo†, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, Bing Yin, Meng Jiang
  ACL 2025. [arxiv]
• [ACL'25] Aligning Large Language Models with Implicit Preferences from User-Generated Content
  Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang
  ACL 2025. [arxiv]
• [NAACL'25] Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
  Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang
  NAACL 2025. [arxiv]
• [NAACL'25] ALERT: An LLM-powered Benchmark for Automatic Evaluation of Recommendation Explanations
  Yichuan Li, Xinyang Zhang, Chenwei Zhang, Mao Li, Tianyi Liu, Pei Chen, Yifan Gao, Kyumin Lee, Kaize Ding, Zhengyang Wang, Zhihan Zhang, Xian Li, Trishul Chilimbi
  NAACL 2025. [paper]
• [NAACL'25] IHEval: Evaluating Language Models on Following the Instruction Hierarchy
  Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, Meng Jiang
  NAACL 2025. [arxiv]
• [EMNLP'25-Findings] Can Language Models Follow Multiple Turns of Entangled Instructions?
  Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji
  EMNLP 2025 (Findings). [arxiv]
• [ArXiv'25] END: Early Noise Dropping for Efficient and Effective Context Denoising
  Hongye Jin, Pei Chen, Jingfeng Yang, Zhengyang Wang, Meng Jiang, Yifan Gao, Binxuan Huang, Xinyang Zhang, Zheng Li, Tianyi Liu, Huasheng Li, Bing Yin
  arXiv 2025. [arxiv]

• [ArXiv'25] Personaagent: When Large Language Model Agents Meet Personalization at Test Time
  Weizhi Zhang, Xinyang Zhang, Chenwei Zhang, Liangwei Yang, Jingbo Shang, Zhepei Wei, Henry Peng Zou, Zijie Huang, Zhengyang Wang, Yifan Gao, Xiaoman Pan, Lian Xiong, Jingguo Liu, Philip S Yu, Xian Li
  arXiv 2025. [arxiv]
• [Information Processing & Management] Leveraging Historical Information to Boost Retrieval-Augmented Generation in Conversations
  Fengran Mo†, Yifan Gao, Zhuofeng Wu, Xin Liu, Pei Chen, Zheng Li, Zhengyang Wang, Xian Li, Meng Jiang, Jian-Yun Nie
  Information Processing & Management. [paper]
• [EMNLP'24] Large Language Models in Clinic: A Comprehensive Benchmark
  Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David A. Clifton
  EMNLP 2024. [arxiv]
• [TMLR'25] Scaling Laws for Predicting Downstream Performance in LLMs
  Yangyi Chen, Binxuan Huang, Yifan Gao, Zhengyang Wang, Jingfeng Yang, Heng Ji
  Transactions on Machine Learning Research, 2025. [arxiv]
• [ICML'24] MEMORYLLM: Toward Self-Updating Large Language Models
  Yu Wang†, Yifan Gao, Xiusi Chen, Haoming Jiang, Shiyang Li, Jingfeng Yang, Qingyu Yin, Zheng Li, Xian Li, Bing Yin, Jingbo Shang, Julian McAuley
  ICML 2024. [arxiv]
• [arXiv'24] Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs
  Kewei Cheng, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Binxuan Huang, Ruirui Li, Shiyang Li, Zheng Li, Yifan Gao, Xian Li, Bing Yin, Yizhou Sun
  arXiv 2024. [arxiv]
• [SIGMOD'24] COSMO: A Large-Scale E-commerce Common Sense Knowledge Generation and Serving System at Amazon
  Changlong Yu, Xin Liu, Jefferson Maia, Tianyu Cao, Yang Li, Yifan Gao, Yangqiu Song, Rahul Goutam, Haiyang Zhang, Bing Yin, Zheng Li
  SIGMOD 2024. [pdf]
• [NeurIPS'24 Benchmark] Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models
  Yilun Jin, Zheng Li, Chenwei Zhang, Tianyu Cao, Yifan Gao, Pratik Jayarao, Mao Li, Xin Liu, Ritesh Sarkhel, Xianfeng Tang, Haodong Wang, Zhengyang Wang, Wenju Xu, Jingfeng Yang, Qingyu Yin, Xian Li, Priyanka Nigam, Yi Xu, Kai Chen, Qiang Yang, Meng Jiang, Bing Yin
  NeurIPS 2024 Benchmark Track. [arxiv]
• [NeurIPS'23] Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns
  Xin Liu, Zheng Li, Yifan Gao, Jingfeng Yang, Tianyu Cao, Zhengyang Wang, Bing Yin, Yangqiu Song
  NeurIPS 2023. [arxiv]
• [EMNLP'23-Findings] Improving Consistency for Text Summarization with Energy Functions
  Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang
  EMNLP 2023 (Findings). [paper]
• [EMNLP'23-Findings] Knowledge-Selective Pretraining for Attribute Value Extraction
  Hui Liu, Qingyu Yin, Zhengyang Wang, Chenwei Zhang, Haoming Jiang, Yifan Gao, Zheng Li, Xian Li, Chao Zhang, Bing Yin, William Yang Wang, Xiaodan Zhu
  EMNLP 2023 (Findings). [paper]
• [arXiv'23] Situated Natural Language Explanations
  Zining Zhu, Haoming Jiang, Jingfeng Yang, Sreyashi Nag, Chao Zhang, Jie Huang, Yifan Gao, Frank Rudzicz, Bing Yin
  arXiv:2308.14115. [arxiv]
• [arXiv'23] CCGen: Explainable Complementary Concept Generation in E-Commerce
  Jie Huang†, Yifan Gao, Zheng Li, Jingfeng Yang, Yangqiu Song, Chao Zhang, Zining Zhu, Haoming Jiang, Kevin Chen-Chuan Chang, Bing Yin
  arXiv:2305.11480. [arxiv]
• [ACL'23] Self-Consistent Chain-of-Thought Distillation
  Peifeng Wang, Zhengyang Wang, Zheng Li, Yifan Gao, Bing Yin and Xiang Ren
  ACL 2023, Outstanding Paper Award. [paper]
• [ACL'23-Findings] Graph Reasoning for Question Answering with Triplet Retrieval
  Shiyang Li†, Yifan Gao, Haoming Jiang, Qingyu Yin, Zheng Li, Xifeng Yan, Chao Zhang and Bing Yin
  ACL 2023 (Findings). [paper]
• [ACL'23-Findings] FolkScope: Intention Knowledge Graph Construction for Discovering E-commerce Commonsense
  Changlong Yu, Weiqi Wang, Xin Liu, Jiaxin Bai, Yangqiu Song, Zheng Li, Yifan Gao, Tianyu Cao, Bing Yin
  ACL 2023 (Findings). [paper]
• [AI Open'23] Improving Task Generalization via Unified Schema Prompt
  Wanjun Zhong, Yifan Gao, Ning Ding, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan
  AI Open, 2023. [paper]
• [RecSys'22] Query Attribute Recommendation at Amazon Search
  Chen Luo, William Headean, Neela Avudaiappan, Haoming Jiang, Tianyu Cao, Qingyu Yin, Yifan Gao, Zheng Li, Rahul Goutam, Haiyang Zhang, Bing Yin
  RecSys 2022 (Industry Talk Proposal). [paper]
• [NAACL'22] ProQA: Structural Prompt-based Pre-training for Unified Question Answering
  Wanjun Zhong*, Yifan Gao*, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan
  NAACL 2022. [arxiv] [paper] [code]
• [NAACL'22-Findings] Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training
  Yifan Gao, Qingyu Yin, Zheng Li, Rui Meng, Tong Zhao, Bing Yin, Irwin King, Michael R. Lyu
  NAACL 2022 (Findings). [arxiv] [paper] [code]
• [arXiv'21] Open-Retrieval Conversational Machine Reading
  Yifan Gao, Jingjing Li, Chien-Sheng Wu, Michael R. Lyu, Irwin King
  arXiv:2102.08633. [arxiv]
• [ACL'21] Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction
  Yifan Gao, Henghui Zhu, Patrick Ng, Cicero Nogueira dos Santos, Zhiguo Wang, Feng Nan, Dejiao Zhang, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
  ACL 2021. [arxiv] [code]
• [EMNLP'20] Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading
  Yifan Gao, Chien-Sheng Wu, Jingjing Li, Shafiq Joty, Steven C.H. Hoi, Caiming Xiong, Irwin King, Michael R. Lyu
  EMNLP 2020. [paper] [code]
• [ACL'20] Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
  Yifan Gao, Chien-Sheng Wu, Shafiq Joty, Caiming Xiong, Richard Socher, Irwin King, Michael R. Lyu, Steven C.H. Hoi
  ACL 2020. [paper] [code]
• [ACL'19] Interconnected Question Generation with Coreference Alignment and Conversation Flow Modeling
  Yifan Gao, Piji Li, Irwin King, Michael R. Lyu
  ACL 2019. [paper] [code]
• [EMNLP'19] Improving Question Generation With to the Point Context
  Jingjing Li*, Yifan Gao*, Lidong Bing, Irwin King, Michael R. Lyu
  EMNLP 2019. [paper]
• [EMNLP'20-Findings] Dialogue Generation on Infrequent Sentence Functions via Structured Meta-Learning
  Yifan Gao, Piji Li, Wei Bi, Xiaojiang Liu, Michael R. Lyu, Irwin King
  Findings of ACL: EMNLP 2020. [arxiv]
• [IJCAI'19] Difficulty Controllable Generation of Reading Comprehension Questions
  Yifan Gao, Lidong Bing, Wang Chen, Irwin King, Michael R. Lyu
  IJCAI 2019. [paper]
• [AAAI'19] Generating Distractors for Reading Comprehension Questions from Real Examinations
  Yifan Gao, Lidong Bing, Piji Li, Irwin King and Michael R. Lyu
  AAAI 2019. [paper] [code]
• [AAAI'19] Title-Guided Encoding for Keyphrase Generation
  Wang Chen, Yifan Gao, Jiani Zhang, Irwin King, Michael R. Lyu
  AAAI 2019. [paper]

• ACL 2023 Outstanding Paper Award, 2023
• Global Top 100 Chinese Rising Stars in Artificial Intelligence, by Baidu Scholar, 2021. [Related News (Chinese)] [Related News]

• Jie Huang, PhD@UIUC, 2022 Summer
• Shiyang Li, PhD@UCSB, 2022 Fall
• Yu Wang, PhD@UCSD, 2023 Summer
• Jiacheng Li, PhD@UCSD, 2023 Fall
• Fengran Mo, PhD@Université de Montréal, 2024 Summer | 2025 Fall
• Scott Chang, PhD@TAMU, 2024 Fall
• Jacky Chan, PhD@HKUST, 2025 Spring
• Zixuan Zhang, PhD@GaTech, 2026 Spring

• Conference Area Chair:
  • ACL/EMNLP/NAACL: 2024, 2025
  • AMLC (Amazon Machine Learning Conference): 2024, 2025
• Journal Reviewer:
  • ACM Transactions on Intelligent Systems and Technology (TIST)
  • Neurocomputing
  • Neural Networks (NN)
  • Natural Language Engineering
  • IEEE/ACM Transactions on Audio Speech and Language Processing (TASLP)
• Conference PC Member:
  • ACL: 2020, 2021, 2022, 2023
  • EMNLP: 2020, 2021
  • NAACL: 2021, 2022
  • ICML: 2020, 2021, 2022, 2023
  • NeurIPS: 2021, 2022, 2023
  • ICLR: 2021, 2022, 2023
  • AAAI: 2019, 2020, 2021, 2022, 2023
  • IJCAI: 2021,2022