AVA-Encoder：面向智能体原生的视频表示学习

View PDF HTML (experimental)

Abstract:Agentic video creation systems remain far from cinematic-grade quality: their underlying models struggle to plan how story, characters, visuals, and sound fit together, and high-quality creation trajectories to learn such planning abilities are scarce. Finished human films embody this knowledge, but in a tightly coupled audiovisual form that agents cannot directly read or manipulate. We propose the Agentic Video Auto-Encoder (AVA-Encoder), which works backward from finished films, turning them into agent-native representations with reconstruction itself as supervision. It encodes a film into a Film Knowledge Graph (Film KG) of structured-text nodes, typed dependency edges, and linked multimodal assets, then regenerates the film from this graph with a fixed decoder. Reconstruction residuals are distilled into textual gradients that drive dual-loop self-evolution: an outer loop learns a shared encoding policy across videos, and an inner loop refines each video's Film KG at test time. AVA-Encoder improves reconstruction fidelity by 20.7 percentage points over the strongest baseline; its learned policy outperforms a finely human-tuned policy with over 70 percent fewer system-prompt tokens; and Film KGs support coordinated editing and improve all tested downstream creation systems. We release the framework, a reconstruction benchmark, and a Film KG dataset.

Subjects:	Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
Cite as:	arXiv:2608.12313 [cs.CV]
	(or arXiv:2608.12313v3 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2608.12313 arXiv-issued DOI via DataCite

Submission history

From: Chuyue Li [view email]
[v1] Wed, 12 Aug 2026 17:58:02 UTC (6,731 KB)
[v2] Tue, 18 Aug 2026 14:28:00 UTC (7,093 KB)
[v3] Tue, 13 Oct 2026 21:47:04 UTC (7,652 KB)