通过稀疏自编码器检索与调控 LLM 高阶语义特征

This paper has been withdrawn by Ruikang Zhang

No PDF available, click to view other formats

Abstract:Recent work in Mechanistic Interpretability (MI) has enabled the identification and intervention of internal features in Large Language Models (LLMs). However, a persistent challenge lies in linking such internal features to the reliable control of complex, behavior-level semantic attributes in language generation. In this paper, we propose a Sparse Autoencoder-based framework for retrieving and steering semantically interpretable internal features associated with high-level linguistic behaviors. Our method employs a contrastive feature retrieval pipeline based on controlled semantic oppositions, combing statistical activation analysis and generation-based validation to distill monosemantic functional features from sparse activation spaces. Using the Big Five personality traits as a case study, we demonstrate that our method enables precise, bidirectional steering of model behavior while maintaining superior stability and performance compared to existing activation steering methods like Contrastive Activation Addition (CAA). We further identify an empirical effect, which we term Functional Faithfulness, whereby intervening on a specific internal feature induces coherent and predictable shifts across multiple linguistic dimensions aligned with the target semantic attribute. Our findings suggest that LLMs internalize deeply integrated representations of high-order concepts, and provide a novel, robust mechanistic path for the regulation of complex AI behaviors.

Comments:	The paper has been substantially restructured, and the new version has been submitted as arXiv:2607.26853
Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2601.02978 [cs.CL]
	(or arXiv:2601.02978v3 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2601.02978 arXiv-issued DOI via DataCite

Submission history

From: Ruikang Zhang [view email]
[v1] Tue, 6 Jan 2026 12:40:37 UTC (621 KB)
[v2] Tue, 7 Apr 2026 02:46:06 UTC (626 KB)
[v3] Tue, 13 Oct 2026 12:15:54 UTC (1 KB) (withdrawn)