CSWBench：评估创意写作中的上下文敏感语码转换

View PDF HTML (experimental)

Abstract:Large language models (LLMs) have demonstrated multilingual capabilities across diverse language generation and understanding tasks. Yet effective code-switching requires more than multilingual proficiency: models must decide when, where, and why to switch languages in ways that fit the surrounding context. This context-sensitive ability remains insufficiently evaluated. We introduce CSWBench, a benchmark for evaluating context-sensitive code-switching in creative writing through two complementary settings: Focused Narrative and Extended Narrative. We further propose CSWEval, a theory-informed framework with two complementary levels: event-level evaluation of Licensing, Contextual Appropriateness, and Functional Gain, and whole-text evaluation of task-specific quality---overall literary and narrative quality in Focused Narrative and quality emphasizing multilingual organization in Extended Narrative. Evaluating twelve LLMs, we find that models exhibit meaningful but uneven code-switching competence across narrative contexts. Model differences are particularly pronounced in multilingual literary integration, while multilingual-focused dimensions show stronger difficulty effects overall than the general narrative dimensions; narratively salient switches are associated with stronger contextual and functional realization; and event-level CS quality is moderately associated with whole-text performance.

Comments:	38 pages, 19 figures, 32 tables. Artifacts: this https URL
Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.20272 [cs.CL]
	(or arXiv:2610.20272v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.20272 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zewen JIang [view email]
[v1] Wed, 14 Oct 2026 14:33:48 UTC (11,255 KB)