PI3D 论文提出针对 3D 环境中多模态大模型的提示词注入攻击

View PDF HTML (experimental)

Abstract:Multimodal large language models (MLLMs) have advanced the capabilities to interpret and act on visual input in 3D environments, empowering diverse applications such as robotics and situated conversational agents. When MLLMs reason over camera-captured views of the physical world, a new attack surface emerges: an attacker can place text-bearing physical objects in the environment to override MLLMs' intended task. While prior work has studied prompt injection in the text domain and through digitally edited 2D images, limited attention has been paid to how these attacks function in 3D environments. To bridge the gap, we introduce PI3D, a prompt injection attack against MLLMs in 3D environments, realized through text-bearing object placement rather than digital image edits. We formulate and solve the problem of identifying an effective pose (position and orientation) for a 3D object with injected text, where the attacker's goal is to induce the MLLM to perform the injected task while ensuring that the object placement remains physically plausible. Experiment results demonstrate that PI3D is an effective attack against multiple MLLMs under diverse camera trajectories. We further evaluate a range of defenses and show that they are not sufficient to reliably defend against PI3D.

Comments:	Conference on Language Modeling (COLM) 2026
Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2602.07104 [cs.CV]
	(or arXiv:2602.07104v3 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2602.07104 arXiv-issued DOI via DataCite

Submission history

From: Zhuoheng Li [view email]
[v1] Fri, 6 Feb 2026 17:19:04 UTC (48,278 KB)
[v2] Wed, 19 Aug 2026 21:32:29 UTC (20,406 KB)
[v3] Thu, 1 Oct 2026 18:15:01 UTC (20,406 KB)