arXiv：cs.AI· Yuhan Li, Zhiyu Jin, Yifan Tong, Wenxiang Shang, Benlei Cui, Xuanhong Chen, Hangcheng Zhu, Bingbing Ni·· 21 小时前
ShoeFit：面向虚拟鞋类试穿的双流 DiT 框架与 MVShoes 数据集
ShoeFit: A New Dataset and Dual-image-stream DiT Framework for Virtual Footwear Try-On
AI 导读
ShoeFit 提出双流 DiT 框架用于虚拟鞋类试穿（VFTON），通过带 3D Rotary Position Embedding 的多视角条件缓解视角错位，并用 LayeredRefAttention 借助背景特征调制鞋类 latent 以减轻背景导致的颜色失真。
正文
Abstract:Virtual footwear try-on (VFTON), a critical yet underexplored area in virtual try-on (VTON), aims to synthesize faithful try-on results given diverse footwear and model images while maintaining 3D consistency and texture authenticity. Unlike conventional garment-focused VTON methods, VFTON presents unique challenges due to (1) Data Scarcity, which arises from the difficulty of perfectly matching product shoes with models wearing the identical ones, (2) Viewpoint Misalignment, where the target foot pose and source shoe views are always misaligned, leading to incomplete texture information and detail distortion, and (3) Background-induced Color Distortion, where complex material of footwear interacts with environmental lighting, causing unintended color contamination. To address these challenges, we introduce MVShoes, a multi-view shoe try-on dataset consisting of 7305 well-annotated image triplets, covering diverse footwear categories and challenging try-on scenarios. Furthermore, we propose a dual-stream DiT architecture, ShoeFit, designed to mitigate viewpoint misalignment through Multi-View Conditioning with 3D Rotary Position Embedding, and alleviate background-induced distortion using the LayeredRefAttention which leverages background features to modulate footwear latents. The proposed framework effectively decouples shoe appearance from environmental interferences while preserving high-quality texture detail through decoupled denoising and conditioning branches. Extensive quantitative and qualitative experiments demonstrate that our method substantially improves rendering fidelity and robustness under challenging real-world product shoes, establishing a new benchmark in high-fidelity footwear try-on synthesis.
| Comments: | Accepted at NeurIPS 2025. Project website: this https URL , including proposed multi-view footwear try-on dataset MVShoes-clean |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18325 [cs.CV] |
| (or arXiv:2610.18325v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18325 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuhan Li [view email]
[v1]
Tue, 13 Oct 2026 13:03:29 UTC (29,656 KB)
来源：arXiv：cs.AI · arxiv.org