MyoSem：将肌电信号与自然语言动作语义对齐

View PDF HTML (experimental)

Abstract:Electromyography (EMG) records muscle activation for hand gesture recognition, prosthetic control, and wearable interaction. Existing EMG methods usually formulate hand action understanding as prediction over fixed gesture labels, leaving EMG signals and action-semantic text unaligned in a common feature space and limiting bidirectional querying between signals and hand-action descriptions. We present MyoSem, an EMG--action semantic alignment framework that maps hand EMG and multi-view natural-language action descriptions into a shared feature space for retrieval. MyoSem builds semantic prototypes from multi-view descriptions, encodes EMG with local activation-aware representations, and uses semantic queries to aggregate local EMG evidence for bidirectional EMG--text retrieval. We evaluate MyoSem under seven protocols on NinaPro DB2, EMG2Pose, and NinaPro DB2$\rightarrow$DB3, covering random splits, user shifts, held-out action classes, and able-bodied-to-amputee transfer. Across this evaluation, MyoSem achieves the strongest overall bidirectional retrieval performance among the evaluated methods. Ablations and representation analyses show how semantic prototypes, activation-aware encoding, and semantic queries shape the learned space. Overall, MyoSem moves EMG hand action understanding from fixed-label recognition toward language-queryable semantic retrieval.

Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
ACM classes:	I.2.6; I.2.7
Cite as:	arXiv:2606.00174 [cs.CV]
	(or arXiv:2606.00174v2 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2606.00174 arXiv-issued DOI via DataCite

Submission history

From: Chiyue Wang [view email]
[v1] Fri, 29 May 2026 13:47:00 UTC (3,398 KB)
[v2] Fri, 9 Oct 2026 06:51:07 UTC (5,184 KB)