神经数据需要语义分词：Tokenization with States 用行为事件划分可跨会话迁移的 token

View PDF HTML (experimental)

Abstract:Extracellular electrophysiology records a different set of neurons in every session. Neural foundation models embed each neuron and each session into their tokens, so every new session is an input they have never seen, and they fail to generalize to it. A tokenizer for new sessions needs a unit that every session shares and that carries behavior. Population activity offers such a unit. It evolves on a low-dimensional manifold that persists across neuronal turnover and across animals once sessions are aligned. This manifold changes regime at task events such as stimulus onset and movement onset. Within each regime the population occupies a state, the part of the manifold it spans between two events, and each state carries its own behavioral meaning. We propose Tokenization with States (TWS), which segments each trial, one repetition of the task, at these events and converts every state into tokens of population geometry, with no neuron or session embedding. On held-out International Brain Laboratory (IBL) sessions, TWS decodes movement even from regime boundaries that carry no information about the target, while a per-neuron foundation model pretrained on those sessions decodes at chance. Frozen after training on mice alone, TWS transfers to macaques and Utah arrays with only a linear probe. On reach direction, a target that its boundaries do not define, it achieves a Matthews correlation of 0.23, where the event time alone achieves $0.01$. For cross-session generalization, the token matters more than the model on top.

Comments:	10 pages, 3 figures
Subjects:	Machine Learning (cs.LG)
Cite as:	arXiv:2610.03001 [cs.LG]
	(or arXiv:2610.03001v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.03001 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Sangyoon Bae [view email]
[v1] Fri, 2 Oct 2026 08:32:36 UTC (1,507 KB)