HuggingFace Daily Papers·· 2 天前
PhysInsert：用几何先验与双向物体-场景交互实现物理合理的物体插入
PhysInsert: Physically Plausible Object Insertion with Geometric Priors and Bidirectional Object-Scene Interactions
AI 导读
PhysInsert 是一个基于参考图像的物体插入框架，可同时适配物体与周围场景。研究团队构建了严格对齐的 Blender 渲染反事实配对数据集 Insert World，并用 Geometry-Guided Adaptation 将 VGGT 几何流与图像、文本流耦合，配合 2D 物体-场景空间监督约束插入位置与环境变化。
正文 · 原文
Abstract:We present PhysInsert, a framework for physically plausible reference-based object insertion that jointly adapts objects and their surrounding scenes. Existing methods often rely on masked inputs, generation-based pairs, or training pairs with limited viewpoint variation, leaving pose and illumination adaptation and bidirectional object-scene interactions insufficiently constrained. We construct the Insert World dataset with strictly aligned Blender-rendered counterfactual pairs and object references across viewpoints and lighting conditions. Building on these data, Geometry-Guided Adaptation couples a VGGT geometry stream with image and text streams under target-scene feature supervision to guide spatially consistent insertion. Complementary 2D Object-Scene Spatial Supervision jointly supervises object occupancy and environmental changes while grounding spatial instructions in object regions through image-text attention. We introduce PhysInsert-Bench with dedicated cross-environment and cross-view cases requiring illumination and pose adaptation beyond appearance copying. Experiments on PhysInsert-Bench and DreamBooth demonstrate improved image fidelity; a user study further shows more accurate placement and improved lighting integration, perspective, and pose plausibility.
| Comments: | 51 pages, including supplementary material. Project page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.40453 [cs.CV] |
| (or arXiv:2610.40453v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40453 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yue Gong [view email]
[v1]
Thu, 29 Oct 2026 17:01:13 UTC (45,370 KB)
来源：HuggingFace Daily Papers · arxiv.org