QuantEval：面向大语言模型金融量化任务的基准测试
QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
QuantEval 是一个评估 LLM 能否完成量化金融"计算—实现—验证"闭环的基准，包含 1,575 个专家验证实例，覆盖知识问答、量化数学推理和量化策略编码三个维度。其编码任务采用确定性 CTA 风格回测框架执行模型生成的策略，并与专家参考的风险收益指标对比。研究评估了 13 个开源与闭源 LLM，发现其与人类专家存在显著差距，并开源了数据集、评测脚本与完整回测配置。
Authors:Zhaolu Kang, Junhao Gong, Wenqing Hu, Yingjie He, Kehan Jiang, Leqi Zheng, Shuo Yin, Zhicheng Fang, Chunlei Meng, Eric Hanchen Jiang, Yunfei Feng, Yitong Leng, Junfan Zhu, Xiaoyou Chen, Xi Yang, Richeng Xuan
Abstract:Large Language Models (LLMs) have shown strong capabilities in finance-oriented question answering, yet their evaluation on professional quantitative workflows remains fragmented and often stops at factual or document-grounded QA. We introduce QuantEval, a benchmark for evaluating whether LLMs can complete the compute--implement--validate loop required in quantitative finance. QuantEval contains 1,575 expert-validated instances across three complementary dimensions: (i) knowledge-based QA, (ii) quantitative mathematical reasoning, and (iii) quantitative strategy coding. The coding task is evaluated with a deterministic CTA-style backtesting framework that executes model-generated strategies and compares risk--return metrics against expert references, moving beyond syntax or template matching. We evaluate 13 open-source and proprietary LLMs, report prompt and reliability analyses, and observe substantial gaps relative to human experts, especially in multi-step reasoning and framework-constrained coding. We further conduct supervised fine-tuning and reinforcement learning experiments on domain-aligned data, showing that training can improve quantitative reasoning while also exposing trade-offs in factual recall. QuantEval releases the dataset, evaluation scripts, and full backtesting configuration to support reproducible research on LLMs for quantitative finance.
| Comments: | this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2601.08689 [cs.CL] |
| (or arXiv:2601.08689v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.08689 arXiv-issued DOI via DataCite |
Submission history
From: Zhaolu Kang [view email]
[v1]
Tue, 13 Jan 2026 16:14:23 UTC (1,627 KB)
[v2]
Fri, 16 Jan 2026 00:38:39 UTC (1,627 KB)
[v3]
Wed, 14 Oct 2026 03:51:15 UTC (1,502 KB)
来源：arXiv：cs.CL · arxiv.org