arXiv：cs.LG· Anubha Gupta, Eduardo Pignatelli·· 3 小时前AI 评分41
LS-AR：自回归 LLM 中的未来预测式潜在引导
LS-AR: Future-Predictive Latent Steering in Autoregressive LLMs
AI 导读
LS-AR 是一种双通道架构，通过 FiLM 条件化将连续目标引导与离散 token 解码解耦。在超出上下文窗口的长程检索任务（H=1024、W=500）中，LS-AR（Static）实现 100% 目标召回，基线为 0%，吞吐量提升约 35%，峰值 VRAM 降低 52.8%。
正文
Abstract:Standard autoregressive (AR) models process high-level task instructions, state history, and transient tokens within a single shared sequence of tokens. Consequently, they lack the architectural mechanisms needed to isolate macro-objectives from context noise. To overcome this single-channel limitation, we introduce Latent-Steered Autoregressive (LS-AR), a dual-channel architecture that decouples continuous goal steering from discrete token decoding via FiLM conditioning. We evaluate a Static Goal Encoder (P_0) for persistent macro-objective retention across long rollouts and a Dynamic State Tracker (P_t) for recurrent latent updates during generation. On long-horizon retrieval past context limits (H=1024, W=500), LS-AR (Static) achieves 100% target recall where parameter-matched baselines collapse (0%), while increasing throughput by ~35% and cutting peak VRAM by 52.8%. In Blocksworld planning under forced perturbations (k=1), LS-AR (Dynamic) sustains an 89.0% completion rate vs. 71.0% for the baseline, though zero-shot entity scaling (N -> N+1) exposes single-vector capacity limits (0%). Finally, dual-channel authority analysis shows that text goal dropout establishes latent-dominant control, offering structural defence against text prompt injection while introducing a latent vector attack surface.
| Comments: | Accepted at the NeurIPS 2026 Workshop: Long-Context Foundation Models |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.03093 [cs.LG] |
| (or arXiv:2610.03093v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03093 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Anubha Gupta [view email]
[v1]
Fri, 2 Oct 2026 10:13:58 UTC (926 KB)
来源：arXiv：cs.LG · arxiv.org