SUN：在强化学习中兼顾新颖性与可达性的目标选择框架

View PDF HTML (experimental)

Abstract:Exploration in reinforcement learning (RL) remains a fundamental challenge. Recent goal-conditioned RL strategies (which select goals to encourage broader state coverage) have shown promising results, but none scores a goal by novelty and reachability jointly: the two signals are traded off by hand, applied in sequence, or one is neglected outright. In this paper, we introduce a reachability-aware goal-selection framework that explicitly integrates these two aspects, and that can be seamlessly incorporated into any off-policy RL algorithm. To this aim, we propose SUccessor-to-Novelty (SUN), an indicator derived from successor value functions to identify goals that are both novel and reachable. We prove that SUN is the value of a count-scaled goal-reaching reward, bounds short-horizon hitting probabilities, and rejects unreachable goals. We further present an adaptive goal-selection strategy that leverages these properties, along with an accurate yet lightweight pseudocount to avoid the overhead of classic methods. We back up all our claims with thorough benchmarks: SUN consistently outperforms state-of-the-art methods in standard and novel environments with unreachable or hard-to-reach states, irreversible transitions, obstacles, mazes, and unbounded spaces. Code at this https URL

Comments:	39 pages. Accepted at the 19th European Workshop on Reinforcement Learning (EWRL 2026)
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Robotics (cs.RO)
Cite as:	arXiv:2609.08642 [cs.LG]
	(or arXiv:2609.08642v2 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2609.08642 arXiv-issued DOI via DataCite

Submission history

From: Wenyan Yang [view email]
[v1] Tue, 8 Sep 2026 12:12:13 UTC (20,404 KB)
[v2] Fri, 9 Oct 2026 07:34:13 UTC (20,260 KB)