arXiv：cs.CL· Chengguang Gan, Yunhao Liang, Hanjun Wei, Qinghao Zhang, Zhixi Cai, Shiwen Ni·· 10 小时前
为什么还要生成 click？为 Web 智能体设计的插件式决策头
Why Generate a Click? Plug-in Decision Heads for Web Agents
AI 导读
研究为基于视觉语言模型的 Web 智能体提出插件式决策头，直接从前向传播的隐藏状态读出动作类型与目标元素，冻结生成器时用 Qwen3-VL 4B 和 8B 在 213k 网页步骤上训练。
正文
Abstract:Web agents built on vision language models decide every action by generating text such as click(12), although most of that text encodes a choice among a few action types and a few dozen marked elements. We read these choices directly from the hidden states that the generator computes while it reads the goal and the screenshot. Plug-in decision heads on a frozen generator predict the action type, pool the visual tokens inside each numbered box to pick the target, and point at a listed text option or at a WRITE slot that leaves the writing of new text to the generator. A confidence cascade executes the heads' action when they are sure and otherwise lets the generator decode from the KV cache of the same forward pass. With Qwen3-VL 4B and 8B generators trained on 213k web steps, a strict threshold keeps the cascade between 0.3 points below and 1.4 points above the generator on seven offline test sets while the heads decide 12 to 64% of the steps. A looser threshold raises step success on unseen Mind2Web websites by 3.1 and 1.8 points, costs up to 2 points on GUIAct and MiniWoB++, and lifts WebArena task success from 12.5 to 13.8% (4B) and from 13.6 to 14.6% (8B) with less than half of the generated tokens. Training the heads jointly with the generator lowers the generator's own step success by up to 4.9 points; heads on a frozen generator avoid this cost.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20107 [cs.CL] |
| (or arXiv:2610.20107v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20107 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chengguang Gan [view email]
[v1]
Wed, 14 Oct 2026 12:39:51 UTC (479 KB)
来源：arXiv：cs.CL · arxiv.org