arXiv：cs.LG· Sajjad Baghaeea, Aimin Lib, Shanmugarajan Balasubramanian, Elif Uysal·· 13 小时前
用强化学习实现深空通信的面向目标 Bundle 管理与流量控制
Goal-Oriented Bundle Management and Flow Control for Deep-Space Communications via Reinforcement Learning
AI 导读
该研究提出一套闭环建模与学习框架，用于接触驱动深空网络中的中继侧 Bundle 调度、丢弃与流量控制，月球与火星更新经共享缓存中继转发至地球。方法以超额信息年龄 Δᵢᵉ(t) 定义面向目标的损失，将中继控制建模为约束半马尔可夫决策过程（CSMDP），并用带掩码的 Lagrangian PPO 求解。DTNSim 仿真显示，其基于年龄的效用与流间公平性优于 FIFO 等基线调度器。
正文
Abstract:Deep-space communications operate under intermittent connectivity, long propagation delays, and relay storage constraints. In such networks, the Bundle Protocol enables store-carry-forward communication, requiring relay nodes to manage limited storage and scarce contact opportunities. Relay-side scheduling and dropping decisions are inherently goal-oriented: transmission resources should be allocated to bundles most valuable to downstream mission objectives. Conventional schedulers, such as FIFO, do not explicitly optimize task utility and may provide unfair service across heterogeneous planetary sources. This paper presents a closed-loop modeling and learning framework for relay-side bundle and flow management in a contact-driven deep-space network, where lunar and Martian updates are forwarded through a shared-buffer relay to Earth. We quantify destination-side information freshness using excess age $\Delta_i^e(t)=[\Delta_i(t)-d_i(t)]^+$, where $\Delta_i(t)$ denotes the Age of Information of source $i$ and $d_i(t)$ is a source-dependent delay baseline. Based on this metric, we define a goal-oriented loss that supports source-dependent nonlinear penalties, allowing prioritization according to mission relevance. We formulate relay control as a constrained semi-Markov decision process (CSMDP) and solve it using masked Lagrangian Proximal Policy Optimization (PPO). Feasibility-aware action masking restricts transmissions to bundles that can complete within the remaining contact window. DTNSim simulations indicate improved age-based utility and inter-flow fairness compared with baseline schedulers. The proposed framework jointly optimizes relay-side bundle scheduling, dropping, and flow control under contact-driven dynamics with explicit fairness considerations across heterogeneous sources.
| Comments: | 15 pages, 3 figures, 4 tables. Presented at the 77th International Astronautical Congress (IAC 2026), Antalya, Turkey, 5-9 October 2026. Paper ID: IAC-26-B2.5.8-x113206 |
| Subjects: | Networking and Internet Architecture (cs.NI); Machine Learning (cs.LG); Systems and Control (eess.SY) |
| Cite as: | arXiv:2610.13923 [cs.NI] |
| (or arXiv:2610.13923v1 [cs.NI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13923 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sajjad Baghaee [view email]
[v1]
Fri, 9 Oct 2026 17:53:46 UTC (2,992 KB)
来源：arXiv：cs.LG · arxiv.org