架构先于公式：Transformer 中依赖训练的计算语义

View PDF HTML (experimental)

Abstract:An operation's input-output function does not, by itself, determine its computational role within a represented process. That role also depends on the processes producing its inputs and receiving its outputs, a philosophical stance termed deep learning externalism (DLE). We develop a mathematical account of represented computation that distinguishes functional equivalence, retained predecessor distinctions, restricted receiver capability, and context-dependent effective architectures. The resulting equivalences depend on which computational relations a scientific description must preserve. We prove that two injective producer contexts can preserve every predecessor distinction while changing whether fixed local and attention receiver families are equivalent.
We investigate these relations in a dimension-labelled PreNorm Transformer and their evolution under training. Residual bypasses restrict complete-state information identifications to two merges per block, while finer branches can change receiver accessibility. Exact gradient-descent models yield covariance-dependent geometric attenuation under learned producer routing and algebraic $(t^{-1/2})$ attenuation under Q/K-softmax training, with optimal bounded-receiver recovery errors derived in both cases. Coupled producer-attention dynamics establish how learning jointly reorganizes the presented features and receiver allocation. Finally, present functional or receiver-profile equivalence need not determine future behavior under an optimizer. In an input-conditioned attention model, two accessibility measurements form an exact reduced predictive state, whereas one alone does not.

Submission history

From: Luis Rosario Freytes [view email]
[v1] Sat, 10 Jan 2026 13:43:01 UTC (62 KB)
[v2] Wed, 22 Jul 2026 02:16:17 UTC (1 KB) (withdrawn)
[v3] Wed, 12 Aug 2026 21:16:43 UTC (32 KB)
[v4] Sun, 11 Oct 2026 01:55:01 UTC (373 KB)