arXiv：cs.LG· Ram Kulathumani, Pushkar Nagar, Regunathan Radhakrishnan, Anupam Tripathi, Xiangbo Mao, Roshanak Omrani, Keshav Somani, Shwet Kamal Mishra, Shayna Lurya·· 6 小时前AI 评分37
EDGE：基于图结构化 DSL 配置的对话模拟确定性图评估引擎
EDGE: Engine for Deterministic Graph Evaluation through Conversation Simulation from Graph Structured DSL Configuration
AI 导读
研究者提出 EDGE 评估方法，基于由 DSL 驱动的规划器 AgentGraph，通过图遍历算法穷举对话路径，构建覆盖智能体完整行为空间的评估集。该方法定义了对响应与轨迹确定性、结构遵循度及语义一致性的新指标，可复现轨迹与 DSL 规范比对。结果显示，使用 AgentGraph 与 LangGraph 等显式结构化节点转换框架配置的智能体，确定性优于未受控转换配置的智能体。
正文
Abstract:As agentic systems evolve into complex multi agent orchestration workflows, there is a growing and critical need for systematic frameworks that measures an agent's behavioral consistency and determinism. In this paper, we introduce a formal evaluation methodology that is grounded in AgentGraph, a planner powered by a domain specific language that represents agent reasoning through a dynamically adjustable directed graph. We leverage this structural formalism and utilize graph traversal algorithms that exhaustively enumerate conversational paths, forming a comprehensive evaluation set that captures the agent's complete behavioral space. We then systematically replay these reproducible trajectories to compare observed outputs and state transitions against the intended DSL specification. To quantify reliability, we define novel metrics that measure response and trajectory determinism, structural adherence and semantic consistency across both exact replays and their linguistic variants. Our system's results demonstrate that agents configured using frameworks like AgentGraph and LangGraph with explicitly structured node transitions show superior determinism over agents that are not configured with controlled transitions.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.29971 [cs.AI] |
| (or arXiv:2608.29971v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.29971 arXiv-issued DOI via DataCite |
Submission history
From: Ram Kulathumani [view email]
[v1]
Sun, 30 Aug 2026 18:56:53 UTC (430 KB)
[v2]
Wed, 7 Oct 2026 05:24:27 UTC (430 KB)
来源：arXiv：cs.LG · arxiv.org