Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition

This study investigates the role of Theory of Mind (ToM) modeling in dialogue-based collaborative plan acquisition (CPA) within a Minecraft environment. While previous research suggested ToM could enhance missing knowledge prediction, particularly for partners with unequal skills, the authors find that improvements due to ToM modeling are limited. Performance in CPA is significantly higher when predicting one's own missing knowledge compared to a partner's, and baseline models with ToM features perform similarly without them. The results indicate that ToM features might capture dataset patterns rather than reflecting true ToM abilities, necessitating a deeper understanding of ToM's role in collaborative AI and the need for alternative methods to model mental states in agents. The study proposes a graph-based representation for dialogue-based CPA, outperforming previous works by representing plans as directed graphs. However, it discovers no significant difference in performance when using ToM features, suggesting that learned features may not directly correspond to mental states. The research contributes by enhancing plan prediction and questioning the current understanding of ToM in collaborative tasks, suggesting that current models might rely on dataset biases. The research combines various aspects, including training models to predict players' mental states, knowledge, and intentions, and examining the Common Partner Awareness (CPA) task. It evaluates a GNN-based model with Graph Attention layers and a Transformer for ToM, using candidate sampling to refine predictions. While the model shows improvements in some tasks, it falls short in accurately capturing temporal aspects and mental states. The study finds that ToM features do not significantly enhance CPA performance, and there is a lack of strong correlation between ToM task performance and CPA. It suggests that current approaches to modeling mental states in collaborative agents might be insufficient and calls for more general world models and interpretable methods. The research is set in the context of Minecraft, a popular game for studying human-like collaboration and problem-solving, and highlights the importance of ethical considerations in AI that model human cognition.