Jerry Liu· @jerryjliu0 · X·· 2 小时前AI 评分49
AI 导读
万圣节快乐！🎃 我们做了一个有趣的基准测试，用来“吓唬”LLM 和 OCR 工具，让它们输出重复字符 很多真实世界的文档中存在重复字符，这会让模型彻底失控 来看看 boobench！ 链接：https://www.llamaindex.ai/benchmarks/boobench 数据集：https://huggingface.co/datasets/llamaindex/BooBench
正文 · 原文
happy halloween! 🎃
we made a fun benchmark to "scare" LLMs and OCR tools into output repeating characters
a lot of real-world documents have repeating characters that cause models to completely go off the rails
check out boobench!
link: https://www.llamaindex.ai/benchmarks/boobench
dataset: https://huggingface.co/datasets/llamaindex/BooBench
We built a haunted house for doc parsers 🎃 Boo Bench is 179 pages that push parsers into repetition loops. One model saw 246 redaction x’s on a page and wrote 23,004 of them. Results in thread! Leaderboard + paper: https://llamaindex.ai/benchmarks/boobench Dataset: https://huggingface.co/datasets/llamaindex/BooBench在 X 查看被引用的帖子
来源：Jerry Liu · x.com