ToM Workshop

About

GENERATIVE AI and ToM IN COMMUNICATING AGENT @ IJCAI 2025

Follow @tom_ICML2024

Motivation

Theory of Mind (ToM) is the ability to reason about the minds of other agents. Although the recent advancements in generative AI have shown ToM and reasoning skills, there are still limitations. The main theme of our workshop is the computational modeling of ToM, with a special focus on the role of natural language in such modeling. Specifically, ToM 2025 will investigate computational modeling of ToM for communication, including challenges and opportunities in the era of LLM along three thrusts:

ToM for ML. Measure and evaluate ToM in ML, benchmarking as human ToM are shaped by beliefs, social norms, interpretability and reasoning of how the ToM-enabled AI systems have come to a decision.

ToM for Cognitive Science (CogSci). ML algorithm design inspired by cognitive science and ToM in humans.

Social Impact and ToM. Evaluate ToM-enabled AI model’s output to the relevance of the user, (e.g., age-appropriate robot interaction), safeguard to mitigate privacy and user risk, hallucination of LLMs.

This workshop intends to promote the community of researchers that are interested in improving the ability of intelligent agents to reason about others' mental states. Our proposed program provides a space to discuss pathways for understanding and applying ToM in psycholinguistics, pragmatics, human value alignment, social good, model explainability, and many other areas of NLP. ToM 2025 will be a full-day hybrid in-person/virtual workshop with several keynote speeches, and oral/poster/spotlight presentations, followed by a breakout discussion, panel discussion, and best paper award announcement. We also intend to host a mentoring program to broaden participation from a diverse set of researchers.

The ToM Workshop will be co-located with IJCAI 2025!

If you have any questions, feel free to contact us.

Calls

Call for papers

We welcome submissions of full papers as well as work-in-progress and accept submissions of work recently published or currently under review.

In general, we encourage three types of papers:

Empirical paper: Submissions should focus on presenting original research, case studies or novel implementations in the fields of machine learning, artificial intelligence, natural language processing, and related areas.
Position paper: Authors are encouraged to submit papers that discuss critical and thought-provoking topics within the scientific community.
Thought pieces: Contributions in this category should provide insights, thought experiments, or discussions pertaining to theoretical or philosophical questions in machine learning and related disciplines. We welcome the papers discussing the relationships between theory-of-mind and language acquisition, large language models, agency, subjectivity, embodiment, AI ethics and safety, social intelligence, artificial intelligence, etc. We also welcome contributions that address related questions or offer new perspectives on existing conversations.

Potential topics include:

Leveraging ToM for Machine Learning Applications (e.g., NLP, Robotics, CV)
Cognitive Science Perspectives of ToM
ToM for HCI / Human-AI collaboration
Surveys or replication of existing work
Social Impacts of ToM

Important Dates

Submission deadline:June 20, 2025
Notification of acceptance: July 14, 2025
Camera-ready papers due: TBD
All deadlines are AoE time.

Submission Guidelines

The ToM 2025 workshop will use CMT as the review platform.

Accepted papers will be presented as posters, and a subset of them will be selected for oral presentation. We plan to organize the ToM workshop at IJCAI 2025 in a hybrid format. For virtual workshop attendees, we plan to use Zoom for the talks/panel and Gather for posters/socializing. To support the hybrid format, we will hold parallel meet-and-greet sessions online and in-person.

The paper template and style files can be found at here (IJCAI Author Kit. Please use LaTeX. ). The length of paper can be as short as 2 pages or as long as 8 pages (excluding references and appendix). Submissions must follow the template and style and should be properly anonymized.

Dual Submission Policy

We welcome papers that have never been submitted, are currently under review, or recently published. Accepted papers will be published on the workshop homepage, but will not be part of the official proceedings and are to be considered non-archival.

Acknowledgment

The Microsoft CMT service was used for managing the peer-reviewing process for this conference. This service was provided for free by Microsoft and they bore all expenses, including costs for Azure cloud services as well as for software development and support.

Accepted Papers

Paper	Title	PDF
1	The Yōkai Learning Environment: Tracking Beliefs Over Space and Time Ruhdorfer et al.	Download
2	Rethinking Theory of Mind Benchmarks for LLMs: Towards A User‑Centered Perspective Wang et al.	Download
3	Bridging the Gap: Unifying HCI & ML Perspectives on Mutual Theory of Mind Ashktorab et al.	Download
4	Theory of Mind in Prisoner’s Dilemma with Small LLMs Gunal et al.	Download
5	Leveraging First‑Person Experience to Infer Third‑Person Beliefs in a Competitive Gridworld Task Michelson et al.	Download
6	MIST: Towards Multi‑dimensional Implicit Bias and Stereotype Evaluation of LLMs via Theory of Mind Li et al.	Download
7	Using Mental Models to Understand the Effect of AI Transparency on Trust Parakh et al.	Download
8	Theory of Mind Needs Privacy: A Collaborative SLM–LLM Framework Luo et al.	Download

Program

🎤 Keynote Speaker

Dr. Vered Shwartz

Assistant Professor of Computer Science

University of British Columbia

Keynote Title: Theory of Mind in Large Language Models: What We’re Testing, What We’re Seeing, and Where We’re Going

Keynote Abstract: Recent years have seen an explosion of research on theory of mind in LLMs. Yet the verdict is still out on whether LLMs have ToM and to what extent. The literature is riddled with conflicting findings, and the surrounding AI hype has often obscured rather than clarified the picture. In this talk, I will address three central questions. First, what are we actually testing? Much of the current work in NLP operates with a narrow notion of ToM, often scratching only the surface of the phenomenon. Second, do LLMs have theory of mind? While LLMs show a degree of understanding in social reasoning tasks, these results are confounded by factors such as data contamination and overly simplistic or artificial experimental designs. Finally, where do we go from here? I will argue for studying a broader set of ToM dimensions, designing benchmarks that simulate realistic human-AI interactions, and integrating insights from neighbouring disciplines.

Bio: Dr. Vered Shwartz is an Assistant Professor of Computer Science at the University of British Columbia, and a CIFAR AI Chair at the Vector Institute. Her research interests focus on natural language processing, with the fundamental goal of building models capable of human-level understanding of natural language. She is interested in computational semantics and pragmatics, commonsense reasoning, culturally-inclusive AI, vision and language, and NLP applications.

Workshop schedule

📅 Date: August 17, 2025

🚪Room 520E

Oral (Long):15 mintues presentation, 5 minutes QA

Oral (Short):12 mintues presentation, 3 minutes QA

Time	Event
09:00 – 09:10	Opening Remarks
09:10 – 09:30	Ruhdorfer et al.: The Yōkai Learning Environment: Tracking Beliefs Over Space and Time
09:30 – 09:50	Wang et al.: Rethinking Theory of Mind Benchmarks for LLMs: Towards A User-Centered Perspective
09:50 – 10:30	Invited Talk: Tomer D. Ullman Genies and Broomsticks: The Intentional and Unintentional Miscommunication of Goals
10:30 – 11:00	☕ Morning Coffee Break
11:00 – 11:30	Invited Talk: Joyce Chai Theory of Mind in Situated Communication
11:30 – 11:45	Ashktorab et al.: Bridging the Gap: Unifying HCI & ML Perspectives on Mutual Theory of Mind
11:45 – 12:00	Gunal et al.: Theory of Mind in Prisoner’s Dilemma with Small LLMs
12:00 – 12:30	Invited Talk: Max Kleiman-Weiner Evolving General Cooperation with a Bayesian Theory of Mind
12:30 – 14:00	🍽️ Lunch
14:00 – 14:50	Keynote: Vered Shwartz Theory of Mind in Large Language Models: What We’re Testing, What We’re Seeing, and Where We’re Going
14:50 – 15:10	Michelson et al.: Leveraging First-Person Experience to Infer Third-Person Beliefs in a Competitive Gridworld Task
15:10 – 15:25	Li et al.: MIST: Towards Multi-dimensional Implicit Bias and Stereotype Evaluation of LLMs via Theory of Mind
15:25 – 15:30	Group Photo
15:30 – 16:00	☕ Afternoon Coffee Break
16:00 – 16:15	Parakh et al.: Using Mental Models to Understand the Effect of AI Transparency on Trust
16:15 – 16:30	Luo et al.: Theory of Mind Needs Privacy: A Collaborative SLM–LLM Framework
16:30 – 16:40	Closing Remarks

Talks

Invited Speakers and Panelists (Tentatively Accepted)

Joyce Y. Chai is a Professor in the Department of Electrical Engineering and Computer Science at the University of Michigan. Her research interests are in the area of natural language processing, situated dialogue agents, human-robot communication, and artificial intelligence. She's particularly interested in language processing that is sensorimotor-grounded, pragmatically-rich, and cognitively-motivated. Her recent work has focused on grounded language processing to facilitate situated communication with robots and other artificial agents.

Tomer Ullman is an Assistant Professor of Psychology at Harvard University, and an affiliate of the Kempner Institute for the study of Natural and Artificial Intelligence. His work focuses on commonsense reasoning and intuitive theories. He has a particular interest in building empirically-grounded computational models of intuitive physics and intuitive psychology.

Vered Shwartz is an Assistant Professor of Computer Science at the University of British Columbia, and a CIFAR AI Chair at the Vector Institute. Her research interests focus on natural language processing, with the fundamental goal of building models capable of human-level understanding of natural language. She is interested in computational semantics and pragmatics, and commonsense reasoning. She is currently working on learning to uncover implicit meaning, which is abundant in human speech, developing machines with advanced reasoning skills, multimodal models, and culturally-aware NLP models.

Max Kleiman-Weiner is an Assistant Professor at the University of Washington and PI of the Computational Minds and Machines lab. His research goal is to create computational models that explain how the mind works and draw on insights from how people learn, think, and act to build smarter and more human-like artificial intelligence.

Pei Zhou is a Senior Applied Scientist at Microsoft Office of Applied Research, where he drives research on improving Copilot reasoning for complex intent, through learning from interaction, synthetic data generation, and UX innovation.