arXiv：cs.AI· Yuxiang Zeng, Kun Xie, Yang Wang, Jigang Wen, Xiaocan Li, Guangxing Zhang, Gaogang Xie, Jiannong Cao·· 22 小时前
DTA-GT：面向神经网络表示学习的方向与拓扑感知图 Transformer
DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning
AI 导读
DTA-GT 是一种基于编码器全局方向一致性的方向与拓扑感知图 Transformer，用于神经网络架构属性预测，在 NAS-Bench-101 和 NAS-Bench-201 的精度与延迟预测上，有限预算设置下持续优于序列、GNN、Transformer 及混合预测器。
正文
Abstract:Architecture attribute predictors reduce the cost of neural architecture search by estimating accuracy or latency from a small set of evaluated candidates. The key difficulty is that neural architectures are operation-labeled DAGs: edge direction defines computational flow, but existing predictors often treat direction as a local structural cue. We propose DTA-GT, a Direction- and Topology-Aware Graph Transformer based on encoder-wide directional consistency, which preserves directed computational semantics across node initialization, global spectral encoding, pairwise interaction, and feature update. DTA-GT realizes this principle with direction- and topology-aware node initialization, Magnetic Laplacian spectral encoding, direction-dual structural attention, and a Direction-Sensitive MoE-FFN. Across accuracy and latency prediction on NAS-Bench-101 and NAS-Bench-201, DTA-GT consistently outperforms representative sequence-, GNN-, Transformer-, and hybrid predictors under limited-budget settings. Ablations and Magnetic spectral controls show that the improvements come from coordinated direction-aware stages and edge-orientation-sensitive spectral information, while mechanistic diagnostics confirm that the modules exhibit the intended directional behavior. These results support encoder-wide directional consistency as an effective design principle for architecture DAG representation.
| Comments: | Accepted at NeurIPS 2026. 36 pages, 6 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE) |
| Cite as: | arXiv:2610.16822 [cs.LG] |
| (or arXiv:2610.16822v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16822 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuxiang Zeng [view email]
[v1]
Mon, 12 Oct 2026 14:04:21 UTC (629 KB)
来源：arXiv：cs.AI · arxiv.org