arXiv：cs.LG· Van Dai Do, Huu Hiep Nguyen, Minh Hoang Nguyen, Manh Nguyen, Hung Le·· 3 小时前
LOTUS：通过预测未来学习理解时序的预训练方法
Lotus: Learning to Understand Time-series by Learning to Forecast
AI 导读
研究提出 forecast grounding 预训练原则，要求语言对齐的时序表征在下游任务适配前保持对未来演化的预测能力，并据此构建 LOTUS。LOTUS 将每个时序通道切分为不重叠 patch 映射进 LLM 嵌入空间，用 LLM 隐藏状态预测下一个 patch，并提出融合原始 patch 与频谱表示的 temporal-spectral patch 表征。
正文
Abstract:Time-series language models typically align numerical signals with language using labels, captions, or question-answer pairs. Although such supervision encourages linguistic representations, it does not explicitly require them to preserve information about how the underlying time-series signal evolves. We introduce forecast grounding, a pretraining principle that requires language-aligned time-series representations to remain predictive of the future evolution before they are adapted to downstream tasks. We instantiate this principle in LOTUS, which divides each time-series channel into non-overlapping patches and maps them into the LLM embedding space. It then predicts the next patch from the LLM hidden states of the time-series tokens, encouraging the learned representation to retain temporal information. To capture both temporal changes and periodic patterns, we also propose a temporal-spectral patch representation that combines each raw time-series patch with its spectral representation before mapping it into the LLM embedding space. Across three language model backbones and six downstream evaluation datasets, LOTUS achieves the best average performance with minimal trainable parameters, reaching 59.91% accuracy and outperforming the strongest matched baseline by 3.36%. Our representation analyses further show that LOTUS uses the numerical input and preserves temporal structure in its learned patch representations. These results establish forecast grounding as a promising pretraining principle, enabling compact time-series language models to outperform models 25 to 50 times their size on TSRBench.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.18877 [cs.LG] |
| (or arXiv:2610.18877v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18877 arXiv-issued DOI via DataCite |
Submission history
From: Van Dai Do [view email]
[v1]
Thu, 1 Oct 2026 01:44:55 UTC (1,075 KB)
来源：arXiv：cs.LG · arxiv.org