语言模型回路究竟告诉我们多少？测量回路的一致性与特异性

View PDF HTML (experimental)

Abstract:The circuits framework in mechanistic interpretability aims to identify sparse subgraphs of model components that are causally responsible for a behavior, typically evaluated by measuring necessity and sufficiency. But these criteria say little about whether a circuit consistently captures how a model performs a task, or if it is specific to that task. We study these two properties, consistency and specificity, across six tasks and five models, extracting circuits at the component level (attention heads and MLP blocks) and at the level of individual MLP neurons. We find that component-level circuits are highly consistent and causally important for most tasks, but they are not specific: for a given task, ablating its own circuit drops accuracy by about as much as ablating another task's circuit. Neuron-level circuits, on the other hand, exhibit higher specificity across tasks, but far less consistency within tasks. This is explained by circuit overlap: component-level circuits share most of their components across tasks, related or not, while neuron-level circuits overlap only between closely related tasks. In a case study of the components shared by the task circuits of Llama-3.2-3B, we show that they consist mostly of MLP blocks, while the few attention heads within turn out to have general-purpose roles. Overall, our findings raise questions about the degree to which circuits can support targeted understanding of, and intervention on, model behavior.

Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2605.08348 [cs.CL]
	(or arXiv:2605.08348v3 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2605.08348 arXiv-issued DOI via DataCite

Submission history

From: Michael Li [view email]
[v1] Fri, 8 May 2026 18:01:03 UTC (6,056 KB)
[v2] Wed, 2 Sep 2026 19:03:08 UTC (9,476 KB)
[v3] Wed, 7 Oct 2026 22:59:12 UTC (1,195 KB)