arXiv：cs.AI· Naiyu Yin, Dennis Wei, Tian Gao, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Yue Yu·· 13 小时前
CircuitLasso：面向大语言模型可扩展电路学习的新方法
Scalable Circuit Learning for Interpreting Large Language Models
AI 导读
CircuitLasso 是一种基于稀疏线性回归的可扩展电路学习方法，用于解读大语言模型内部机制。它在基准数据上的结构准确率与最先进的干预式方法相当，计算成本却大幅降低，并能高效揭示 SAE 特征之间的关联。该研究成果已被 EMNLP 2026 主会议接收。
正文
Abstract:A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit learning methods computationally prohibitive. We propose CircuitLasso, a scalable circuit learning approach based on sparse linear regression. CircuitLasso recovers circuits whose structural accuracy matches that of state-of-the-art intervention-based methods on the benchmark data, at a fraction of the computational cost. For interpretability, CircuitLasso efficiently uncovers relationships among SAE features, showing how human-interpretable semantic features propagate through the model and influence its predictions. Finally, we validate the utility of our learned circuits by leveraging their insights to achieve comparable performance at substantially lower cost on a domain-generalization task.
| Comments: | Accepted to the Main Conference of EMNLP 2026 (The 2026 Conference on Empirical Methods in Natural Language Processing) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.16939 [cs.LG] |
| (or arXiv:2606.16939v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2606.16939 arXiv-issued DOI via DataCite |
Submission history
From: Naiyu Yin [view email]
[v1]
Mon, 15 Jun 2026 16:40:43 UTC (996 KB)
[v2]
Fri, 9 Oct 2026 01:39:55 UTC (983 KB)
来源：arXiv：cs.AI · arxiv.org