用大语言模型检测临床试验中的 Spin 问题

View PDF HTML (experimental)

Abstract:Spin in clinical trials includes reporting practices that distort the presentation of results. This is particularly critical in medicine, where spin is present in more than 50% of randomized controlled trials that fail to reach statistical significance. The comparison of primary and reported outcomes is crucial for detecting several types of spin, including outcome switching. We used 300 pairs of outcomes labeled with semantic similarity to develop a system for automatic detection of outcome switching. We evaluated baseline text similarity models and open-source LLMs using generated similarity scores and the Youden index to determine the classification threshold. The proposed approach involves prompt engineering, classification based on token probabilities, and majority voting for the final decision. The results on the test set of 2,496 examples with an F1 score of 0.78 and an accuracy of 0.90 outperform baseline text similarity models but trail behind fine-tuned versions of BERT. We used LLMs to generate natural language explanations for the classified instances and manually assessed their quality.

Comments:	5 pages, 1 figure, 2 tables. Accepted at the 29th International Multiconference Information Society (IS 2026), AI in Healthcare track, Ljubljana, Slovenia. Code: this https URL
Subjects:	Computation and Language (cs.CL)
ACM classes:	I.2.7; J.3
Cite as:	arXiv:2610.11845 [cs.CL]
	(or arXiv:2610.11845v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.11845 arXiv-issued DOI via DataCite (pending registration)
Related DOI:	https://doi.org/10.70314/is.2026.aihc.42 DOI(s) linking to related resources

Submission history

From: Tjaš Ajdovec [view email]
[v1] Thu, 8 Oct 2026 12:33:52 UTC (79 KB)