arXiv：cs.LG· Jiahao Jiang·· 4 小时前AI 评分31
Memory Prediction Excess：随机过程中预测增益与记忆长度的概率度量
Memory Prediction Excess: A Probabilistic Quantity for Predictive Gain and Memory Length in Stochastic Processes
AI 导读
研究者提出 Memory Prediction Excess（MPE），用于量化离散时间有限状态过程中，利用完整历史相比仅用静态边缘分布带来的预测准确率平均提升。MPE 恒为非负，存在依赖静态准确率的上界，并给出归一化版本作为预测效率的无量纲度量。框架扩展至有限长度历史（FH-MPE），导出达到与完整历史同等预测性能所需的最小记忆长度，并证明有限阶马尔可夫链中该长度受马尔可夫阶数约束。
正文
Abstract:A central question in the prediction of stochastic processes is the extent to which past information can improve the probability of correctly predicting the next state. We introduce the Memory Prediction Excess (MPE) to address this question quantitatively. The MPE measures the average improvement in prediction accuracy obtained by using the entire observed history relative to using only the static marginal distribution, in discrete-time finite-state processes. It is defined as the difference between the expected optimal conditional prediction accuracy and the optimal static prediction accuracy. Its basic properties are examined: the MPE is always non-negative; it admits an upper bound depending on the static accuracy, attained if and only if the future is almost surely a deterministic function of the past; and degenerate cases in which the MPE vanishes are characterized. A normalized version, taking values in the unit interval, is introduced as a dimensionless measure of predictive efficiency. A lower bound is derived by comparing predictions based on histories of different lengths, showing that the expected optimal prediction accuracy is monotone with respect to the history length. The framework is extended to finite-length histories, where the finite-history MPE (FH-MPE) measures the predictive gain attainable when only the most recent observations are retained. This leads to the notion of a minimal memory length required to achieve the same predictive performance as the full history. For finite-order Markov chains, this minimal memory length is shown to be bounded by the Markov order. The MPE and its variants are formulated in terms of conditional probabilities and prediction accuracies, offering a probabilistic perspective on the predictive utility of memory that is complementary to classical information-theoretic approaches.
| Subjects: | Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG); Probability (math.PR) |
| MSC classes: | 60G07, 62M20, 60J10, 68T05, 94A17 |
| Cite as: | arXiv:2610.06894 [stat.ML] |
| (or arXiv:2610.06894v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06894 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiahao Jiang [view email]
[v1]
Mon, 28 Sep 2026 08:52:12 UTC (45 KB)
来源：arXiv：cs.LG · arxiv.org