arXiv：cs.AI· Ojas Nimase, Zhe Chen, Yue Zhao, Xiyang Hu·· 18 小时前
GEO-Bench：生成引擎优化中的排序操纵基准测试
GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimization
AI 导读
研究者推出 GEO-Bench，首次在同一协议下统一评测生成引擎优化（GEO）的排序操纵攻击。该基准涵盖黑盒提示攻击（TAP、Zero-Shot）、白盒梯度攻击（STS、RAF、StealthRank）及十种白帽 C-SEO 策略，在五个数据集上以固定开源排序器 Llama-3.1-8B-Instruct 打分。结果显示有效性与隐蔽性存在权衡，黑盒内容改写可媲美甚至超越梯度攻击且更难被检测。
正文
Abstract:Large language models (LLMs) increasingly rank products, documents, and recommendations for user queries, which makes manipulating these rankings a growing concern for fairness and information integrity. Research on generative engine optimization (GEO) has produced many manipulation methods, but each is evaluated on its own dataset with its own metrics, so their relative strength and detectability stay unclear. We present GEO-Bench, a benchmark that evaluates GEO ranking-manipulation attacks under one protocol. It unifies black-box prompt-based attacks (TAP, Zero-Shot), white-box gradient-based attacks (STS, RAF, StealthRank), and ten white-hat C-SEO strategies. We score every method on five datasets against a fixed open-weight ranker (Llama-3.1-8B-Instruct), using metrics for both effectiveness (NRG, Success@{\alpha}, Promote@{\alpha}) and stealth (keyword violation rate, perplexity ratio). Our evaluation shows that effectiveness and stealth trade off across adversarial attacks, that black-box content rewriting matches or exceeds gradient-based attacks on rank promotion while producing more fluent text and can evade both keyword- and perplexity-based detection on some domains, and that the access model does not predict attack strength. By standardizing datasets, attack implementations, and metrics, GEO-Bench enables the first direct comparison across these attack paradigms and supports the development of detection methods.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.29107 [cs.CR] |
| (or arXiv:2605.29107v3 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2605.29107 arXiv-issued DOI via DataCite |
Submission history
From: Ojas Nimase [view email]
[v1]
Wed, 27 May 2026 21:10:43 UTC (59 KB)
[v2]
Sat, 30 May 2026 18:12:24 UTC (59 KB)
[v3]
Mon, 12 Oct 2026 06:01:08 UTC (162 KB)
来源：arXiv：cs.AI · arxiv.org