ELSA3D：面向统一 3D 理解与生成的弹性语义锚定模型

View PDF HTML (experimental)

Abstract:Unified 3D foundation models aspire to generate 3D assets and reason about them in language within a single backbone, but their text-3D interaction remains largely implicit. Existing methods concatenate text and 3D tokens into a flat sequence and rely on self-attention, collapsing coarse structural cues and fine geometric details into one undifferentiated representation. We introduce ELSA3D, a unified 3D model that addresses this with elastic semantic anchoring, structuring language and geometric reasoning jointly along matched abstraction scales. ELSA3D represents geometry with a scale-aware octree tokenizer and introduces Anchor Tokens, sparse cross-modal units that select semantic cues, route them to the most relevant 3D scale, retrieve scale-specific geometric evidence, and write the fused signal back into the unified representation, keeping interaction sparse yet precise. A lightweight per-block router makes both computation and reasoning elastic, choosing which text tokens instantiate anchors at which geometric scale so that cross-modal capacity concentrates where alignment is most needed. ELSA3D achieves state-of-the-art performance across image-to-3D generation, text-to-3D generation, and 3D captioning, outperforming the strongest unified baseline while roughly halving FLOPs and inference latency relative to the non-elastic version of the same model.

Comments:	Accepted at NeurIPS 2026. Project link: this https URL
Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as:	arXiv:2607.06565 [cs.CV]
	(or arXiv:2607.06565v2 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2607.06565 arXiv-issued DOI via DataCite

Submission history

From: Tianjiao Yu [view email]
[v1] Tue, 7 Jul 2026 17:59:50 UTC (26,519 KB)
[v2] Thu, 1 Oct 2026 18:32:32 UTC (18,658 KB)