arXiv：cs.AI· Farnaz Soleimani (LISSI), Abdelghani Chibani (LISSI), Yacine Amirat (LISSI), Ghazaleh Khodabandelou (LISSI)·· 16 小时前
Neuro-Symbolic Graph Transformer 用于多模态人体活动序列的分层意图识别
Neuro-Symbolic Graph Transformers for Hierarchical Intention Recognition from Multimodal Human Activity Sequences
AI 导读
研究提出 Neuro-Symbolic Graph Transformer（NSGT），通过将 Human Reasoning Graph 以可学习拓扑偏置融入自注意力机制，并加入可微一阶逻辑损失作为结构正则项，实现原子动作、复合活动、低层意图和高层意图四层本体一致性。
正文
Abstract:Proactive human-robot collaboration requires autonomous systems to infer not only immediate actions but also the underlying hierarchical intentions governing human behavior. Current deep learning approaches predominantly focus on flat action classification, failing to capture the structural dependencies connecting kinematic motion to high-level goals. A Neuro-Symbolic Graph Transformer (NSGT) is proposed within this multimodal framework, a Human Reasoning Graph (HR-G) is integrated into the self-attention mechanism via learnable topological biases. Furthermore, the architecture is augmented with a differentiable First-Order Logic (FOL) loss, acting as a structural regularizer by which ontological consistency is enforced across four levels: atomic actions, composite activities, Low-Level Intentions, and High-Level Intentions. Evaluation is performed on the synthesized NTU-Hierarchy dataset, a four-level ontological extension of NTU RGB+D 120, wherein existing sequences were mapped into 119,383 complex hierarchical episodes (85 actions, 16 activities, 8 LLIs, 4 HLIs). To prevent circular supervision, strict isolation between label generation and FOL constraints was maintained. In the base configuration, 91.65% top-1 action accuracy is achieved by the NSGT. When the FOL loss is incorporated into the baseline, logic consistency is improved by 3.26 percentage points to 96.67%. Through the HR-G biases, a tunable trade-off between accuracy and structural coherence is established. In the primary consistency-maximizing configuration ($\lambda$ = 0.5), a logic consistency of 94.59% is reached and ontological contradictions are reduced by 33%, while a robust 89.90% top-1 action accuracy is retained. Finally, interpretable reasoning evidence is provided by the learned HR-G weights, wherein transition edges are strongly exploited ($\beta$ trans $\approx$ +2.6) without explicit supervision.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.18176 [cs.AI] |
| (or arXiv:2610.18176v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18176 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Farnaz SOLEIMANI [view email] [via CCSD proxy]
[v1]
Tue, 13 Oct 2026 11:20:51 UTC (1,615 KB)
来源：arXiv：cs.AI · arxiv.org