arXiv：cs.AI· Xiaowei Su, Xusen Cheng·· 8 小时前
PatTransBench：面向专利机器翻译评测的 IPC 分层多维基准
PatTransBench: an ipc-stratified, multi-dimensional benchmark for patent machine translation evaluation
AI 导读
PatTransBench 是一个 12,000 条的中英/日/韩专利机器翻译基准，包含 6,000 条短文本与 6,000 条长文本对，按 IPC 分类分层采样并覆盖三个语向的双向翻译。
正文
Abstract:We present PatTransBench, a 12,000-item Chinese-English/Japanese/Korean patent machine translation (PatMT) benchmark with a four-dimensional evaluation protocol. The benchmark comprises 6,000 short-text (sentence/claim) and 6,000 long-text (paragraph) pairs, evenly split across three language pairs and both translation directions, built via IPC-section-stratified sampling and dualthreshold quality gating from upstream patent-family parallel corpora. The evaluation protocol spans literal n-gram overlap (BLEU, ROUGE-L, chrF), cross-lingual semantic textual similarity (CLSTS), 38-probe-language consistency testing, and serving performance under concurrency. A case study on PatMT2-7B and its fp8/AWQ4/AutoRound-4bit variants reveals direction-dependent quantization effects masked by aggregate scores and non-monotonic language-failure patterns across quantization schemes that are invisible to single-metric evaluation. We plan to release the benchmark, protocol, and case-study results upon publication to support reproducible PatMT evaluation.
| Comments: | 4 pages, 2 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16538 [cs.CL] |
| (or arXiv:2610.16538v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16538 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaowei Su [view email]
[v1]
Mon, 12 Oct 2026 10:25:06 UTC (56 KB)
来源：arXiv：cs.AI · arxiv.org