arXiv：cs.AI（全量分类）· Julie Krugler Hollek, Michael Zargham, Mala Kumar·· 5 小时前AI 评分26
基于本体的情境化 AI 评估（OB-CAIE）方法论
Ontology-Based Contextual AI Evaluations (OB-CAIE) Methodology
AI 导读
OB-CAIE 方法论通过领域本体（DSO）与评估过程本体（EPO）明确界定 AI 评估的测试范围，以解决测试覆盖不清、可复现性不足的问题。该方法在需要人类判断的环节引入科学化的人类反馈，并支持在规范问题空间内追踪、可视化与分析失败点。
正文
Abstract:The ontology-based contextual AI evaluation (OB-CAIE) methodology was developed to address a lack of scientific rigor that arises from unclear testing coverage, to balance human expertise and automations, and to address a lack of reproducibility of AI evaluation testing environments. OB-CAIE strengthens the current state of AI evaluations by addressing the first step in the scientific method by clearly defining what will be tested. Two ontologies represent the tractable problem space in the OB-CAIE methodology: the Domain-Specific Ontology (DSO) and the Evaluation Process Ontology (EPO). The DSO is the what; the EPO is the how. An OB-CAIE problem space can be used for one or multiple AI evaluations. The OB-CAIE methodology allows for human judgment at specific points, in scientifically grounded ways, and in complex subject areas where human feedback is genuinely irreducible or machine irreplaceable. A key advantage of the OB-CAIE methodology is that failure points can be traced, visualized and analyzed within the canonical OB-CAIE methodology problem space.
| Comments: | 17 pages, 2 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| ACM classes: | I.2.4; D.2.5; D.2.1 |
| Cite as: | arXiv:2610.00529 [cs.AI] |
| (or arXiv:2610.00529v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00529 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mala Kumar [view email]
[v1]
Wed, 30 Sep 2026 18:15:41 UTC (658 KB)
来源：arXiv：cs.AI（全量分类） · arxiv.org