HuggingFace Daily Papers·· 2 天前
任意 3D 场景仅需 1K Tokens：面向大规模场景生成的 3D 锚定表征
Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale
AI 导读
该研究提出直接在由 DINOv2、SigLIP2、DA3 等 2D 基础模型衍生的高维隐式 3D 潜空间中生成 3D 场景，并用扩散 Transformer 在该空间建模，标题称任意 3D 场景仅需 1K Tokens。实验显示该方法在效率与空间一致性上优于基于 2D 的方法，并验证了其在表征与模型容量上的可扩展性。论文已被 NeurIPS 2026 接收，代码已开源。
正文 · 原文
Abstract:Scene-level 3D generation has long been dominated by 2D multi-view or video diffusion models. Typically, these 2D-based approaches perform scene generation in a 2D latent space, which introduces two fundamental issues: (i) representing 3D scenes via 2D views leads to significant representation redundancy, and (ii) latent space rooted in 2D inherently limits the spatial consistency of the generated scenes. In this paper, we propose to perform 3D scene generation directly within a high-dimensional implicit 3D latent space derived from powerful 2D foundation models (e.g., DINOv2, SigLIP2, DA3). Then we tame diffusion transformers to perform diffusion modeling directly in the 3D latent space, enabling 3D-grounded scene generation. In our experiments, we not only demonstrate the advantages of our method over 2D-based approaches in terms of efficiency and spatial consistency, but also comprehensively analyze the impact of 3D latent representation on 3D scene generation performance. Furthermore, we validate the favorable scalability of our method with respect to representation and model capacity.
| Comments: | Accepted at NeurIPS 2026. Code has been released. Project Page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computational Geometry (cs.CG) |
| Cite as: | arXiv:2604.11331 [cs.CV] |
| (or arXiv:2604.11331v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.11331 arXiv-issued DOI via DataCite |
Submission history
From: Dongxu Wei [view email]
[v1]
Mon, 13 Apr 2026 11:32:36 UTC (34,594 KB)
[v2]
Mon, 19 Oct 2026 15:46:05 UTC (42,902 KB)
来源：HuggingFace Daily Papers · arxiv.org