arXiv：cs.AI· Kapil Wanaskar, Gaytri Jena, Vinija Jain, Aman Chadha, Amitava Das·· 20 小时前
ECLIPTICA：通过 CITA 实现可切换 LLM 对齐的框架
ECLIPTICA -- A Framework for Switchable LLM Alignment via CITA - Contrastive Instruction-Tuned Alignment
AI 导读
ECLIPTICA 将 LLM 对齐变为指令驱动、运行时可控制，通过自然语言对齐指令实时调节行为，并提出 CITA 结合 SFT 与对比偏好优化。在 Llama-3.1-8B 的五个测试套件上，CITA 达到 86.7% 的指令对齐效率，超过 DPO（56.1%）、GRPO（36.1%）和 PPO（20.4%）。
正文
Abstract:Alignment in large language models (LLMs) is still largely static: after training, the policy is frozen. DPO, GRPO methods typically imprint one behavior into the weights, leaving little runtime control beyond prompt hacks or expensive re-alignment. We introduce ECLIPTICA, which treats alignment as instruction-driven and runtime-controllable: natural-language alignment instructions act as an explicit behavioral contract (stance, refusal boundary, verbosity) that modulates behavior on the fly under evolving safety requirements, user roles, and governance constraints.
We introduce CITA (Contrastive Instruction-Tuned Alignment), combining SFT with contrastive preference optimization under an explicit geometric anchor to a reference model. This yields a stable Riemannian chart and keeps instruction updates within a shared neighborhood, so regimes stay nearby and traversable for reliable switching.
To isolate policy switching from ordinary instruction following, we release the ECLIPTICA benchmark: 3000 controlled cases (300 prompts x 10 instruction types) where the user request is fixed and only the alignment instruction changes. On Llama-3.1-8B across five suites (ECLIPTICA, TruthfulQA, Conditional Safety, Length Control, LITMUS), CITA reaches 86.7% instruction-alignment efficiency, beating DPO (56.1%), GRPO (36.1%), and PPO (20.4%).
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.06157 [cs.LG] |
| (or arXiv:2601.06157v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.06157 arXiv-issued DOI via DataCite |
Submission history
From: Vinija Jain [view email]
[v1]
Tue, 6 Jan 2026 08:28:26 UTC (3,372 KB)
[v2]
Sun, 11 Oct 2026 20:38:41 UTC (3,372 KB)
来源：arXiv：cs.AI · arxiv.org