arXiv：cs.AI· Xuan Wen, Yonghao Song, Yuhong Zhang, Guoliang Li, Tzyy-Ping Jung, Xiaorong Gao·· 13 小时前
LaS：用语言监督训练神经解码器，语音解码准确率提升至多 17.15pp
Teaching neural decoders with language
AI 导读
研究提出 Language as Supervision（LaS），通过可学习投影将神经特征与自然语言类别描述对齐，并用候选文本原型完成分类。在头皮与颅内记录的视觉、运动、语音解码中，LaS 语音解码准确率较匹配的 CLS 基线最高提升 17.15pp。其表示支持候选集复用与受标签监督的被试和会话适配，并可在 SSVEP 响应中实现条件性未见频率解码。
正文
Abstract:Brain-computer interfaces decode neural activity, yet class-label supervision (CLS) leaves relationships among classes implicit. We propose Language as Supervision (LaS), which aligns neural features with natural-language class descriptions through learned projections and predicts classes using candidate text prototypes. We evaluate seven encoder architectures with independently trained models for visual, motor, and speech decoding from scalp and intracranial recordings. LaS improves speech-decoding accuracy by up to 17.15~pp over matched CLS baselines. Its representations support candidate-set reuse and label-supervised subject and session adaptation, while LaS enables conditional unseen-frequency decoding in steady-state visual evoked potential (SSVEP) responses. Correct class-description correspondence supports this generalization, and dual-sided attribution visualization shows neural and textual contributions to the same class-matching score. These findings suggest that text descriptions can serve as general-purpose references for learning and interpretation in neural decoding.
| Comments: | 14 pages, 8 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.12921 [cs.AI] |
| (or arXiv:2610.12921v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12921 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yonghao Song [view email]
[v1]
Fri, 9 Oct 2026 03:12:42 UTC (8,275 KB)
来源：arXiv：cs.AI · arxiv.org