AcrossVAM2.0：冲突感知点监督实现参数高效的视频扩散模型适配

View PDF

Abstract:Explicit point correspondence is attractive for adapting video generators: it describes how visible surfaces move, but can overwrite the appearance prior learned by a large pretrained model. We study this tension in a 5.01B-parameter text-to-video diffusion transformer. ACROSSVAM2.0 combines a confidence-weighted track-and-visibility head, zero-initialized residual feedback, and asymmetric PCGrad that removes only the auxiliary component opposing the primary flow-matching gradient on shared LoRA parameters. The design preserves the generator exactly at initialization and protects its primary update during adaptation. In a four-way ablation trained for 1,000 steps with three independent seeds, the full method lowers KLT forward-backward cycle error in all three seeds relative to a matched flow-only continuation (mean seed-level difference -0.00711, SD 0.00505), while robust sharpness remains positive in all three. An expanded 24-prompt evaluation (216 videos) yields 9.7% lower aggregate CoTracker3 cycle error than the matched control and near-matched performance on six VBench dimensions; however, the seed-level tracking effect is heterogeneous and I3D-FVD regresses from 0.1824 to 0.2022. These results support a bounded claim: conflict-aware adaptation improves correspondence consistency without broad perceptual degradation, but does not uniformly improve video quality. We release equations, pseudocode, matched outputs, source tables, and artifact provenance.

Submission history

From: Nan Wu [view email]
[v1] Sat, 10 Oct 2026 08:12:08 UTC (4,164 KB)