Introducing the Document Parsing Protocol (DPP)

There are hundreds of different OCR models and tools, but they might all parse a document in different ways.

Some errors are very clear-cut (e.g. hallucinating text, or completely misaligning a table), but many errors are much more ambiguous (e.g. how to represent split tables sharing a header). It's an issue that plagues most parse benchmarks too, because they inherently reward a certain style even if other styles might be semantically correct.

We introduce a protocol that proposes a standard way of parsing these documents into a correct format. We've curated a list of 140 rules that cover literally every aspect of doc parsing, for instance:
- tables: borderless whitespace tables, header-only tables, horizontal slicing, side-by-side mini tables
- forms: key-value regions, checkboxes, checklists, ids and labels, fields outside the form,
- charts: standardized chart output, multiple charts, gantt charts, line charts

You can use DPP as an evaluation rubric for LLMs when determining the quality of parsing a document.

It's not a full evaluation benchmark (yet), but an initial evaluation guide, and meant to encourage a discussion on what are the "best" ways to parse a document.

Check it out: https://www.parseprotocol.com/guide/forms-forms-and-key-value-regions

Blog: https://www.llamaindex.ai/blog/introducing-the-document-parsing-protocol