arXiv：cs.AI· Zhankai Ye, Bofan Li, Yukai Jin, Shuoqiu Li, Wei Wang, Yanfu Zhang, Shangqian Gao, Xin Liu·· 22 小时前
GeoMotionGPT：用大语言模型实现几何对齐的运动理解
GeoMotionGPT: Geometry-Aligned Motion Understanding with Large Language Models
AI 导读
GeoMotionGPT 通过让运动码本与 LLM 嵌入空间共享正交几何基，替代以往仅靠 token ID 连接量化与语义嵌入的做法。该方法在 HumanML3D 上聚合 Average 较最强基线提升 22.4%，在 KIT-ML 上提升 14.4%，消融实验验证了 tokenizer、投影与正则化设计的有效性。
正文
Abstract:Discrete motion tokenization has recently enabled Large Language Models (LLMs) to serve as versatile backbones for motion understanding and motion-language reasoning. However, existing pipelines typically decouple motion quantization from semantic embedding learning, linking them solely via token IDs. This approach fails to effectively align the intrinsic geometry of the motion space with the embedding space, thereby hindering the LLM's capacity for nuanced motion reasoning. We argue that alignment is most effective when both modalities share a unified geometric basis. Therefore, instead of forcing the LLM to reconstruct the complex geometry among motion tokens from scratch, we present a novel framework that explicitly enforces orthogonality on both the motion codebook and the LLM embedding space, ensuring that their relational structures naturally mirror each other. Specifically, we employ a decoder-only quantizer with Gumbel-Softmax for differentiable training and balanced codebook usage. To bridge the modalities, we use a sparse projection that maps motion codes into the LLM embedding space while preserving orthogonality. Finally, a two-stage orthonormal regularization schedule enforces soft constraints during tokenizer training and LLM fine-tuning to maintain geometric alignment without hindering semantic adaptation. Extensive experiments show that our framework improves the aggregated Average by 22.4% over the strongest baseline on HumanML3D and by 14.4% on KIT-ML, while ablations confirm the effectiveness of the tokenizer, projection, and regularization designs.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.07632 [cs.CV] |
| (or arXiv:2601.07632v5 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2601.07632 arXiv-issued DOI via DataCite |
Submission history
From: Zhankai Ye [view email]
[v1]
Mon, 12 Jan 2026 15:14:29 UTC (4,947 KB)
[v2]
Wed, 14 Jan 2026 02:19:38 UTC (6,694 KB)
[v3]
Mon, 16 Mar 2026 13:36:38 UTC (6,815 KB)
[v4]
Thu, 19 Mar 2026 13:41:22 UTC (6,815 KB)
[v5]
Mon, 12 Oct 2026 13:50:11 UTC (1,940 KB)
来源：arXiv：cs.AI · arxiv.org