arXiv：cs.AI· Seyedarmin Azizi, Arya Fayyazi, Parsa Razmara, Erfan Baghaei Potraghloo, Souvik Kundu, Massoud Pedram·· 6 小时前
激活引导的曲率分析：尖锐方向为何让 CAA 失效
The Sharp Directions Are Against You: Curvature Analysis of Activation Steering
AI 导读
研究提出"steering Hessian"来分析激活引导向量几何，其顶部特征向量为尖锐方向，微小扰动即导致行为剧烈波动；尖锐方向仅占 CAA 向量平方范数的 1-5%，却使行为反向。
正文
Abstract:Activation steering adds a vector to a language model's internal representations at inference time. It is a lightweight alternative to fine-tuning for behavioral control. The standard construction, contrastive activation addition (CAA), is fragile, as on many inputs it shifts behavior in the wrong direction. To explain this fragility, we first study the geometry of the steering vector. We introduce steering Hessian, the matrix of second order derivative of the model's behavioral loss with respect to the steering vector. It captures the output behavior response sensitivity of the model under small perturbations of the steering vector along each direction. Its top eigenvectors are sharp directions: tiny perturbations along them can cause large, erratic, input-dependent behavioral swings. Bottom eigenvectors are flat directions: behavior is robust along them. Removing the sharp component of a steering vector approximately preserves its direction and magnitude (cosine > 0.974, norm reduction < 3% for typical settings) while reducing the vector's directional curvature by up to 12x (median 4x across 36 conditions). We evaluate on three model families (Qwen2.5-7B, Llama-3.1-8B, Mistral-7B), with four behavioral tasks, and 36 token-level conditions. Steering along sharp directions reverses the intended behavior in every condition tested, with anti-steer rates of 53-100%. Sharp directions account for only 1-5% of the CAA vector's squared norm. However, they drive a disproportionate share of the gap between the steered and unsteered next-token distributions during autoregressive generation. We term this one-line correction as sharp removal; it outperforms standard CAA in open-ended generation across all 25 conditions tested. A jailbreaking experiment further shows that sharp directions act as noise rather than signal: they disrupt coherent behavioral control regardless of steering polarity.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16084 [cs.AI] |
| (or arXiv:2610.16084v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16084 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Parsa Razmara [view email]
[v1]
Mon, 12 Oct 2026 02:49:19 UTC (249 KB)
来源：arXiv：cs.AI · arxiv.org