When we say Parseloom achieves 94% extraction accuracy on invoices, that number needs a definition. Without a definition, accuracy claims for document extraction are nearly meaningless.
What the number is measuring
Our 94% figure is field-level accuracy: the percentage of extracted fields whose values exactly match the ground truth across our benchmark set. We define "exact match" strictly. For numeric fields, the extracted value must match to the cent. For date fields, it must match to the day and use a normalized format. For text fields, it must match after case normalization and whitespace stripping.
This is stricter than many published accuracy numbers, which use fuzzy string matching or count a result as correct if it contains the right value anywhere in the output.
What's in the benchmark set
Our invoice benchmark has 1,840 invoices collected from early-access teams across 14 vendor types. We used actual vendor documents, not synthetically generated ones. Each invoice was manually annotated by two reviewers with disagreements resolved by a third. The set includes multi-page invoices (about 12% of the total), invoices with non-standard layouts, and some documents with poor scan quality.
We re-run the benchmark every two weeks when the extraction model changes. Results are tracked in an internal dashboard. If a model update drops accuracy below 92% on any field category, we don't ship it.
Where the 6% failure comes from
Most failures fall into three categories. First, layout novelty: invoices from vendors we've never seen, where the field positions don't match any trained template. These account for roughly half the failures. Second, table parsing: line-item tables with non-standard column structures. Third, scan quality: documents that were faxed or photocopied below 150 DPI where character recognition degrades.
Failures are not distributed randomly. A single new vendor's invoice layout can contribute 20-30 failures to one benchmark run. Adding that vendor to the training set eliminates those failures. This is why teams see accuracy improve over time as they use the product.
Why this matters for your use case
If you're processing invoices from 5 well-established vendors, your accuracy will be higher than 94% because those vendor layouts are well-represented in our training data. If you're processing documents from 40 different vendors, some of them rare, you should expect a lower baseline and plan to review the first batch of extractions from each new vendor type before automating downstream steps.
The confidence score on each extraction helps here. Fields with confidence below 0.75 are more likely to be wrong. Routing those for human review catches most errors before they reach the ERP.