arXiv：cs.CL· Kyusong Lee, Heting Ying, Tiancheng Zhao·· 16 小时前
VLX-Go：通过多阶段学习与同策略精炼实现鲁棒具身跟踪
VLX-Go: Robust Embodied Tracking through Multi-Stage Learning and On-Policy Refinement
AI 导读
VLX-Go 是一种视觉-语言路点策略，先在 19,532 条 oracle 轨迹上完成视觉-语言对齐、多模态推理中期训练与跟踪专门化，将干扰物跟踪成功率从 OmTrackVLA 的 41.54% 提升至 55.80%。
正文
Abstract:Embodied visual tracking requires a robot to identify a designated target from visual observations and control its own motion to follow that target through an environment. Following an isolated person is a comparatively simple demonstration of embodied visual tracking, but following the intended person among distractors remains challenging. Existing methods exhibit a substantial drop in success from single-target to distractor tracking, motivating a focus on the shared spaces that person-following robots must navigate. We present \method{}, a vision--language waypoint policy built around the hypothesis that visual and language representations should be aligned before tracking-specific motion is learned. The policy combines complementary frozen visual encoders, multiscale temporal context, a transformer backbone, and a lightweight head for robot-centric waypoint prediction. Its training progresses through vision--language alignment, multimodal reasoning mid-training, and tracking specialization on 19,532 oracle episodes. Compared with 41.54\% distractor-tracking success for our earlier OmTrackVLA model, the complete multi-stage recipe achieves 55.80\%. We then explore on-policy refinement using filtered oracle labels for states visited by the policy. This preliminary study adds 2.42 percentage points of distractor-tracking success, reaching 58.22\%, while reducing collision rate by 1.99 pts. The final policy records the highest distractor- and ambiguous-tracking success rates among the compared methods. These results support staged multimodal learning as a promising direction for tracking among distractors and motivate further study of on-policy refinement for person following in shared spaces.
| Comments: | tech report |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL); Robotics (cs.RO) |
| Cite as: | arXiv:2610.19361 [cs.CV] |
| (or arXiv:2610.19361v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19361 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tiancheng Zhao [view email]
[v1]
Wed, 14 Oct 2026 01:02:16 UTC (4,577 KB)
来源：arXiv：cs.CL · arxiv.org