arXiv：cs.LG· Ken Nakahara, Aleksei Buvailik, Prokhor Kotov, Roberto Calandra·· 4 小时前AI 评分44
面向灵巧抓取稳定性的时序视触觉学习
Temporal Visuo-Tactile Learning for Dexterous Grasp Stability
AI 导读
研究团队用配备四个 Digit 360 触觉传感器的多指机械手采集了 200 个物体、10,000 次抓取的数据集，涵盖视觉、本体感觉与触觉流，训练端到端时序多模态模型从提拉前观测预测提拉后稳定性。
正文
Abstract:Humans can grasp everyday objects with almost perfect success rates using fingertip tactile feedback, yet much of the robotic grasping literature emphasizes vision-based grasp selection with parallel grippers. In this work, we systematically investigate how high-resolution, dynamic tactile sensing contributes to grasp stability prediction and model-guided grasping in dexterous robotic hands. To this end, we collected a dataset of 10,000 grasp trials across 200 objects using a multi-fingered robotic hand equipped with four Digit 360 tactile sensors, recording external vision, proprioception, and tactile streams throughout each grasp. With this dataset, we trained end-to-end temporal multimodal models to predict post-lift stability from pre-lift grasp observations and compared sensing modalities and encoding backbones. Experimental results and controlled input ablations show that incorporating touch, and particularly high-resolution, dynamic touch, improves grasp stability prediction. Finally, we deployed the learned predictor as an online stability gate on the real robot, where visuo-tactile model-guided regrasping improved the success rate among executed lifts by 10.5 percentage points over a non-tactile gate. These results show how rich fingertip sensing and expressive temporal models that capture the dynamics of touch can support learned grasping with multi-fingered hands without explicit contact or force modeling, providing a scalable data-driven path from tactile experience toward stable dexterous manipulation. The dataset is publicly available at this https URL.
| Comments: | 12 Pages. Website: this https URL |
| Subjects: | Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.10283 [cs.RO] |
| (or arXiv:2610.10283v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10283 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ken Nakahara [view email]
[v1]
Wed, 7 Oct 2026 15:48:19 UTC (2,036 KB)
来源：arXiv：cs.LG · arxiv.org