Hi, I am Sitao Cheng, a Ph.D. student at the University of Waterloo, fortunate to be advised by Prof. Victor Zhong. I am currently a Student Researcher at Samaya AI, working with Yuhao Zhang. Before that, I was a research scholar in the UCSB NLP Group with Prof. William Wang, and a research intern at Microsoft Research Asia. I received my Master’s degree from Nanjing University.

My research asks how reasoning in large language models can generalize efficiently: how a model can autonomously compose the skills it already has (parametric) with newly acquired ones (contextual), and recursively improve itself. I work across language agents, reinforcement learning, retrieval-augmented generation (RAG), and neural-symbolic reasoning. Three questions drive my current work:

1. Efficient generalization of language agents. How can an agent build a reusable hierarchy of skills from its own experience, and how can training teach it to capture how those skills compose?
2. RL and compositional generalization. Which training strategies produce models that compose skills they were never explicitly taught, and what does that reveal about how RL works?
3. Automatic reward modeling. Can reward functions be discovered rather than hand-designed, by a reward proposer that learns from its own trials?

I am always glad to talk about research, so please feel free to reach out. You can also read my CV.

Recent News

Preprints

X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalization

Sitao Cheng, Xunjian Yin, Zhiyuan Sun, Yuxuan Li, Ruiwen Zhou, Xiangru Jian, Victor Zhong

Preprint, 2026

Paper Project page Blog Code

From Atomic to Composite: Reinforcement Learning Enables Generalization in Complementary Reasoning

Sitao Cheng, Xunjian Yin, Ruiwen Zhou, Yuxuan Li, Xinyi Wang, Liangming Pan, William Yang Wang, Victor Zhong

Preprint, 2025

Paper Project page Code Data

Dual-Evolving Reinforcement Learning for Automated Reward Design

Sitao Cheng*, Tianle Li*, Xuhan Huang*, Xunjian Yin, Difan Zou

* Equal contribution

Preprint, 2026

Paper Project page Code Models

Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systems

Ruiwen Zhou*, Maojia Song*, Xiaobao Wu, Sitao Cheng, Xunjian Yin, Yuxi Xie, Zoey Hao, Wenyue Hua, Liangming Pan, Soujanya Poria, Min-Yen Kan

* Equal contribution

Preprint, 2026

Paper Code

Selected Publications

Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models

Sitao Cheng, Liangming Pan, Xunjian Yin, Xinyi Wang, William Yang Wang

KnowFM Workshop @ ACL 2025 Oral

Paper Code

Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environments

Sitao Cheng, Ziyuan Zhuang, Yong Xu, Fangkai Yang, Chaoyun Zhang, Xiaoting Qin, Xiang Huang, Ling Chen, Qingwei Lin, Dongmei Zhang, Saravan Rajmohan, Qi Zhang

Findings of ACL 2024

Paper Code

LEDOM: Reverse Language Model

Xunjian Yin, Sitao Cheng, Yuxi Xie, Xinyu Hu, Li Lin, Xinyi Wang, Liangming Pan, William Yang Wang, Xiaojun Wan

ACL 2026 Oral

Paper Model

QueryAgent: A Reliable and Efficient Reasoning Framework with Environmental Feedback based Self-Correction

Xiang Huang*, Sitao Cheng*, Shanshan Huang, Jiayu Shen, Yong Xu, Chaoyun Zhang, Yuzhong Qu

* Equal contribution

ACL 2024 Oral

Paper Code

Disentangling Memory and Reasoning Ability in Large Language Models

Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, Yongfeng Zhang

ACL 2025

Paper Code

EfficientRAG: Efficient Retriever for Multi-Hop Question Answering

Ziyuan Zhuang, Zhiyang Zhang, Sitao Cheng, Fangkai Yang, Jia Liu, Shujian Huang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang

EMNLP 2024

Paper Code

RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios

Ruiwen Zhou, Wenyue Hua, Liangming Pan, Sitao Cheng, Xiaobao Wu, En Yu, William Yang Wang

ACL 2025

Paper Code

Teaching

• CS240 Data Structures and Data Management, Teaching Assistant, Winter & Spring 2026, UWaterloo
• CS115 Introduction to Computer Science 1, Teaching Assistant, Spring 2025, UWaterloo

Services

• Reviewer: ARR, ICLR, ICML, NeurIPS
• Volunteer: ACL 2024