Guided Action Flow：面向冻结 VLA 策略的价值引导采样方法

Authors:Liuhaichen Yang, Zhuang Jiang, Chenchao Sheng, Ningwei Bai, Qichen Yin, Hanbo Ma, Junkai Liu, Junkai Sun, Dongcheng Lyu, Yi Dong, Zezhi Tang

View PDF HTML (experimental)

Abstract:Reinforcement learning can improve vision-language-action (VLA) policies beyond supervised fine-tuning, although this typically involves further updates to the policy parameters. For flow-matching policies, iterative action generation provides an additional opportunity to incorporate task information during inference. We introduce Guided Action Flow (GAF), which learns a compact, observation-conditioned action-value critic from robot task rollouts and applies its action gradient to steer reverse-time flow sampling. The supervised-fine-tuned VLA remains frozen throughout critic learning and deployment. Physical-robot experiments show an increase in aggregate success from 60.0% to 82.5% across six nominal manipulation tasks. Under six altered-lighting and object-distractor conditions evaluated on three of these tasks, aggregate success improves from 34.2% to 49.2%. Ablations and rollout analyses support the importance of the learned guidance direction and the critic's visual and proprioceptive inputs. With approximately 2.735M trainable critic parameters alongside a 0.45B-parameter VLA, GAF enables task outcomes to inform action generation through a compact inference-time guidance module.

Submission history

From: Liuhaichen Yang [view email]
[v1] Thu, 2 Jul 2026 12:30:50 UTC (1,292 KB)
[v2] Fri, 3 Jul 2026 08:21:35 UTC (1,292 KB)
[v3] Sun, 23 Aug 2026 20:50:03 UTC (1,854 KB)
[v4] Tue, 6 Oct 2026 13:43:17 UTC (5,499 KB)