arXiv：cs.CL· Soyoung Oh, Nursulu Sagimbayeva, Nicholas Jennings, Paulina Hussein, Mehwish Alam, Vera Demberg·· 16 小时前
用强化学习让 LLM 学会感知到行动的规则，提升心智理论推理
Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning
AI 导读
研究用 GRPO 训练 Qwen2.5 和 Llama3，通过生成感知到行动的 if–then 规则来提升心智理论（ToM）推理，在四个 ToM 基准上优于提示词、监督微调和 think 推理。≥3B 模型生成的规则还能迁移，提升 GPT-4o-mini 的下游表现，表明显式规则可弥合 ToM 推理差距并支持跨模型知识迁移。
正文
Abstract:In Theory-of-Mind (ToM) reasoning, large language models may correctly infer the agent's mental state but fail to use that inference when predicting or judging the agent's subsequent actions. In this work, we investigate whether explicitly connecting perception, belief, and action with rules improves ToM reasoning. We generate concise \textit{if--then} rules that express relations between perception to action and use Group Relative Policy Optimization (GRPO) to train Qwen2.5 and Llama3 models across four ToM benchmarks. GRPO with rule generation outperforms prompting, supervised fine-tuning, and think reasoning. Moreover, rules generated by larger models ($\geq$3B) provide transferable knowledge that improves downstream performance of GPT-4o-mini. These findings suggest that explicit rules help models bridge the reasoning gap in ToM and support knowledge transfer across models.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19796 [cs.AI] |
| (or arXiv:2610.19796v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19796 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Soyoung Oh [view email]
[v1]
Wed, 14 Oct 2026 08:40:06 UTC (884 KB)
来源：arXiv：cs.CL · arxiv.org