arXiv：cs.AI· Yingxuan Cheng, Tong Li, Yayong Li, Yuchuan Fan, You-Gan Wang, Jinran Wu, Scott Chapman, Geoffrey J. McLachlan·· 9 小时前
澳大利亚基准测试：LLM 评估土壤有机碳的可靠性如何？
How Reliable Are Large Language Models for Soil Organic Carbon Assessment? An Australian Benchmark
AI 导读
一项澳大利亚基准测试评估了 7 种通用 LLM 配置在土壤有机碳（SOC）评估上的表现，任务包括估算 308 个牧场的 0–10 cm SOC 浓度、计算 120 个情景的 SOC 储量及比较 300 对牧场。
正文
Abstract:Soil organic carbon (SOC) is an indicator of agricultural soil condition and carbon storage, but numerical outputs from large language models (LLMs) require validation before use in monitoring or management. We evaluated seven general-purpose LLM configurations on three direct-response tasks: estimating 0--10 cm SOC concentrations for 308 Australian paddocks under two information conditions, calculating SOC stock in 120 matched scenarios, and comparing 300 paddock pairs. References were measured SOC records or deterministic calculations; response validity, repeated-call dispersion and estimated API cost were assessed separately. Additional measured soil attributes reduced concentration mean absolute error by 13--62\% across configurations, although high-SOC underprediction and paddock-specific errors persisted. Six configurations were highly accurate in stock calculations using equation-native units without coarse fragments, but accuracy declined with physically equivalent alternative units. Pairwise accuracy was 90.5--94.6\% overall and 59.1--70.9\% for 52 closely separated pairs; some configurations were sensitive to presentation order. Nearly all outputs were parseable, without necessarily being correct. The benchmark identifies three distinct requirements for AI-mediated SOC assessment: agreement with measured concentrations, unit-consistent stock calculations and reliable discrimination between sites. These findings support task-specific validation and auditable calculations, rather than unverified use of LLM outputs as ecological evidence.
| Subjects: | Performance (cs.PF); Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2610.18201 [cs.PF] |
| (or arXiv:2610.18201v1 [cs.PF] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18201 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jinran Wu [view email]
[v1]
Tue, 13 Oct 2026 11:37:20 UTC (3,915 KB)
来源：arXiv：cs.AI · arxiv.org