arXiv：cs.CL· Youshan Zhang, Jialu Li·· 11 小时前
基于时延系统的序列建模新框架，实现内容感知的历史召回
Time Delay System for Sequence Modeling
AI 导读
研究者提出一种基于非线性时延系统的序列建模框架，其有效时延依据当前输入与历史输入的语义相似度动态估计。该框架包含可微语义滞后估计器、具闭式累积和求解的对角状态空间（每步 O(1) 复杂度、无顺序循环），以及可解释的模糊语义调制机制，在文本分类与生成任务上优于现有 SOTA 方法。
正文
Abstract:Existing sequence models share a critical limitation: they lack an explicit, interpretable mechanism for content-aware temporal recall. LSTMs compress everything into a single hidden state, thereby losing information about specific past events. Transformers attend to all tokens indiscriminately at quadratic cost, while state-space models like Mamba compress history into a fixed-dimensional state without explicit content-addressable memory. To address this gap, we introduce a novel framework grounded in nonlinear time-delay systems, in which the effective delay is dynamically estimated based on the semantic similarity between current and historical inputs. Our approach features three key innovations: (i) a differentiable semantic lag estimator that computes content-aware soft attention weights over history, producing a weighted historical context and similarity score; (ii) a diagonal state space with closed-form cumulative sum solution, achieving O(1) per-step complexity with no sequential loops; and (iii) an interpretable fuzzy semantic modulation mechanism that learns context-dependent rules for balancing instantaneous processing versus historical recall, where rule centers correspond to prototypical semantic contexts. Experimental results on text classification and generation outperform state-of-the-art methods.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.12864 [cs.CL] |
| (or arXiv:2610.12864v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12864 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Youshan Zhang [view email]
[v1]
Fri, 9 Oct 2026 01:43:37 UTC (583 KB)
来源：arXiv：cs.CL · arxiv.org