TimeWarp：评估 Web Agent 能否适应不断演变的网页

View PDF

Abstract:As web agents close the gap with humans on benchmarks, one question arises: Do today's agents perform just as well on tomorrow's web? We introduce TimeWarp, a benchmark that emulates the evolving web. TimeWarp consists of three web environments, each with six UI versions spanning UI design, frontend code, and workflows from different eras of the internet. We pair TimeWarp with a set of complex, realistic tasks covering different forms of web navigation. Our experiments reveal that vision-based agents are vulnerable to changes, while text-based agents become brittle once fine-tuned on a single version. To address this, we propose TimeTraj, a new annotation method that uses plan distillation to collect trajectories across multiple versions. By training agents on teacher rollouts using our BC-variant, we achieve substantial performance gains: 20.4% to 37.7% for Qwen-3 4B and 0% to 27.0% for Llama-3.1 8B models. Our work helps study generalization across web designs and opens a new paradigm for collecting plans rather than trajectories to improve the robustness of web agents.

Comments:	NeurIPS 2026 Evaluations and Datasets
Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Cite as:	arXiv:2603.04949 [cs.AI]
	(or arXiv:2603.04949v4 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2603.04949 arXiv-issued DOI via DataCite

Submission history

From: Md Farhan Ishmam [view email]
[v1] Thu, 5 Mar 2026 08:43:06 UTC (11,625 KB)
[v2] Tue, 8 Sep 2026 06:29:33 UTC (11,827 KB)
[v3] Sat, 12 Sep 2026 06:26:13 UTC (11,829 KB)
[v4] Wed, 14 Oct 2026 10:16:21 UTC (11,775 KB)