arXiv：cs.AI· Seunghyun Oh, Hirotaka Hiraki, Shuyue Stella Li, Yulia Tsvetkov, Shyamnath Gollakota·· 6 小时前AI 评分49
Jarvis：面向多方对话的主动式语音智能体
Jarvis: A Proactive Speech Agent for Multi-Party Conversations
AI 导读
Jarvis 是一款实时主动式语音智能体，能在多人对话中旁听并在群体遗漏或说错事实且几轮内未自行纠正时发声介入，所有主张均以事先共享文档中的源句为依据。研究同时提出 CHI-180-proactive 合成多方数据集与基于小规模开源权重模型的主动式骨干。该系统对多数介入事件回答正确，在群体自行解决问题时保持沉默的比例达 97%，23 人实地研究验证了上述趋势。
正文
Abstract:Speech agents are reactive and dyadic: they speak when spoken to, and to one person at a time. We ask what it takes for a speech agent to instead take part in a conversation among several people and speak up only when it can help. We introduce Jarvis, a real-time proactive speech agent that audibly participates in multi-party human conversations. Grounded in a document shared beforehand, Jarvis follows the discussion and intervenes when the group misses or misstates a fact and does not correct itself within a few turns. We make three contributions: a problem setting based on epistemic breakdowns that makes proactive intervention measurable, realized as CHI-180-proactive, a synthetic multi-party dataset seeded with known gaps, errors, and self-corrections; a proactive backbone that harnesses a small, open-weight model with deterministic checks and grounds every claim in a source sentence; and interaction techniques for taking the floor in live speech and showing the cited evidence on screen. On CHI-180-proactive, Jarvis is correct on most events it addresses and stays silent 97% of the time when the group resolves an issue itself. A live study with 23 participants confirms these trends with real-time interventions.
| Comments: | 45 pages, 6 figures, 20 tables |
| Subjects: | Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.07506 [cs.HC] |
| (or arXiv:2610.07506v1 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07506 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Seunghyun Oh [view email]
[v1]
Mon, 5 Oct 2026 23:18:04 UTC (3,388 KB)
来源：arXiv：cs.AI · arxiv.org