arXiv：cs.LG（机器学习，全量分类）· Gadiel Sznaier Camps, Chengyang He, Guillaume Sartoretti, Eduardo Montijano, Mac Schwager·· 13 小时前AI 评分32
ScaffoldM3C：面向生成式稳定建筑规划的多模态序列蒙特卡洛框架
ScaffoldM3C: A Multimodal Sequential Monte Carlo Framework for Generative Stable Construction Planning
AI 导读
ScaffoldM3C 是一个多模态、轻量级自回归模型，用于稳定的积木式建筑规划，通过引入辅助脚手架 token 显式考虑脚手架对中间结构的稳定作用。该模型比同类基线小 4 倍，推理速度提升 5 至 20 倍，建筑质量与 SOTA 方法相当且整体稳定性更高。研究将 StableText2Brick 数据集扩展为包含图像条件提示与脚手架稳定建造序列，并通过仿真和真实机器人装配演示验证了效果。
正文
Abstract:Autonomously constructing physically realizable 3D structures remains a significant challenge due to combinatorial action spaces, interchangeable components, equifinal assembly sequences, and strict stability requirements during construction. State-of-the-art methods fine-tune large language models for text-based generative construction. However, these approaches do not allow for Multimodal (text, image, sketch) conditioning, overlook the practical role of scaffolding for stabilizing intermediate structures, and suffer from slow inference speeds. Therefore, we formulate construction as a probabilistic next-block generation task with multiple potential assembly actions and multiple potential task conditioning modalities. Concurrently, we explicitly consider the utility of scaffolding by introducing an auxiliary scaffold block token. We present Scaffold Multimodal Monte Carlo (ScaffoldM3C), a multimodal, lightweight, auto-regressive model for stable block-based construction, that proposes a set of next-step candidate blocks. Leveraging these candidates, we utilize Sequential Monte Carlo (SMC) to maintain a population of possible assembly sequences, allowing us to consider multiple, potentially different, assembly directions simultaneously. We train our multimodal architecture by extending the StableText2Brick dataset to contain image conditioning prompts and scaffold-stabilized build sequences. ScaffoldM3C is 4x smaller than competing baselines, yielding a 5x to 20x speedup during inference, while achieving comparable construction quality to state-of-the-art methods and higher overall stability. We demonstrate the effectiveness of our approach through simulations and real-world robot assembly demonstrations.
| Comments: | This work has been submitted to IEEE Transactions on Robotics and Learning (T-RL) and is currently under review. Project Page: this https URL |
| Subjects: | Robotics (cs.RO); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00487 [cs.RO] |
| (or arXiv:2610.00487v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00487 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Gadiel Sznaier Camps [view email]
[v1]
Wed, 30 Sep 2026 18:00:17 UTC (7,643 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org