olmOCR-Bench

Name: olmOCR-Bench Benchmark Results
Creator: Allen Institute for AI
License: https://creativecommons.org/licenses/by/4.0/

Allen Institute for AI

7,010 unit tests across 1,402 PDF documents. Tests parsing of tables, math, multi-column layouts, old scans, and more.

Models16

Papers28

Metrics9

SOTA History

Coming Soon

Visual timeline of state-of-the-art progression over time will appear here.

Percentage of unit tests passed

Higher is better

Rank	Model	Code	Score	Paper / Source
1	chandra-ocr-0.1.0 7,010 unit tests across 1,402 PDF documents. #1 overall on olmOCR-Bench.	HF	83.1%	AlphaXiv
2	infinity-parser-7b	-	82.5%	AlphaXiv
3	olmocr-v0.4.0		82.4%	AlphaXiv
4	paddleocr-vl		80%	AlphaXiv
5	dots-ocr-3b		79.1%	GitHub
6	mistral-ocr-3 Estimated based on 74% win rate vs OCR 2	-	78%	mistral-announcement
7	marker-1.10.0		76.5%	GitHub
8	marker-1.10.1		76.1%	AlphaXiv
9	deepseek-ocr	-	75.7%	AlphaXiv
10	deepseek-ocr Chandra outperforms by 7.7 points	-	75.4%	GitHub
11	mineru-2.5		75.2%	AlphaXiv
12	mistral-ocr-api	-	72%	AlphaXiv
13	gpt-4o-anchored GPT-4o with anchored prompting	-	69.9%	GitHub
14	nanonets-ocr2-3b	-	69.5%	AlphaXiv
15	gemini-flash-2	-	63.8%	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	dots-ocr-3b #1 on table recognition		88.3	GitHub
2	chandra-ocr-0.1.0 Table recognition category. Near-best (dots.ocr: 88.3)	HF	88	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	chandra-ocr-0.1.0 Mathematical notation in old scans. #1, leads by 5.4 points	HF	80.3	GitHub
2	olmocr-v0.3.0 #2 on math in old scans		79.9	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	chandra-ocr-0.1.0 Long documents with tiny text. #1 in category	HF	92.3	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	chandra-ocr-0.1.0 Base clean document parsing. Near-perfect	HF	99.9	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	olmocr-v0.3.0 #1 on headers/footers extraction		95.1	GitHub
2	chandra-ocr-0.1.0 Header/footer extraction	HF	90.8	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	chandra-ocr-0.1.0 Multi-column document parsing	HF	81.2	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	marker-1.10.0 #1 on ArXiv paper parsing		83.8	GitHub
2	chandra-ocr-0.1.0 ArXiv paper parsing. Marker leads (83.8)	HF	82.2	GitHub

Higher is better

Rank	Model	Code	Score	Paper / Source
1	chandra-ocr-0.1.0 Old scan recognition. #1 (GPT-4o: 40.7)	HF	50.4	GitHub
2	gpt-4o #2 on old scans. Chandra leads by 9.7 points	-	40.7	GitHub