Scene Text Detection2020en

icdar-2013

Dataset from Papers With Code

Metrics:accuracy, cer, wer, f1

Legacy BenchmarkLast significant update: Jan 2019

Legacy benchmark from 2013. For current OCR evaluation, use OCRBench, ICDAR 2019/2021, or DocVQA.

Recommended alternatives:

OCRBench v2 ICDAR 2019 Arbitrary-Shaped Text ICDAR 2015 Incidental Scene Text

Current State of the Art

TrOCR-large 558M

Unknown

98.4

accuracy

accuracy Progress Over Time

Showing 3 breakthroughs from Jul 2015 to Sep 2021

Key Milestones

Jul 2015

CRNN

Lexicon-free. Table 2. TPAMI 2017.

86.7

Source

Mar 2021

ABINet-LV

ABINet Language-Vision variant. CVPR 2021.

97.0

+11.9%

Source

Sep 2021

TrOCR-large 558MCurrent SOTA

TrOCR-large, Syn+Benchmark training. Table 6. AAAI 2023.

98.4

+1.4%

Source

Total Improvement

13.5%

Time Span

6y 3m

Breakthroughs

Current SOTA

98.4

Top Models Performance Comparison

Top 5 models ranked by accuracy

Best Score

98.4

Top Model

TrOCR-large 558M

Models Compared

Score Range

11.7

accuracyPrimary

#	Model	Score	Paper / Code	Date
1	TrOCR-large 558M	98.4	TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models	Sep 2021
2	TrOCR-base 334M	98.4	TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models	Sep 2021
3	PARSeqOpen Source Research	98.13	Scene Text Recognition with Permuted Autoregressive Sequence Models	Jul 2022
4	ABINet-LVOpen Source Fang et al.	97	Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition	Mar 2021
5	CRNN	86.7	An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition	Jul 2015

f-measure

#	Model	Score	Paper / Code	Date
1	TextFuseNet (ResNeXt-101)	94.61	TextFuseNet: Scene Text Detection with Richer Fused FeaturesCode	May 2020
2	SPCNET	92.1	Scene Text Detection with Supervised Pyramid Context Network Code	Nov 2018
3	Mask TextSpotter	91.7	Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes Code	Jul 2018
4	WordSup (VGG16-synth-icdar)	90.34	WordSup: Exploiting Word Annotations for Character based Text Detection	Aug 2017
5	STN-OCR	90.3	STN-OCR: A single Neural Network for Text Detection and Text Recognition Code	Jul 2017
6	PixelLink+VGG16 2s MS	88.1	PixelLink: Detecting Scene Text via Instance Segmentation Code	Jan 2018
7	Corner Localization (multi-scale)	88	Multi-Oriented Scene Text Detection via Corner Localization and Region Segmentation Code	Feb 2018
8	TextBoxes++_MS	88	TextBoxes++: A Single-Shot Oriented Scene Text Detector Code	Jan 2018
9	Corner-based Region Proposals	87.6	Detecting Multi-Oriented Text with Corner-based Region Proposals Code	Apr 2018
10	SSTD	87	Single Shot Text Detector with Regional Attention Code	Sep 2017
11	SegLink	85.3	Detecting Oriented Text in Natural Images by Linking Segments Code	Mar 2017
12	Gupta et al.	83	Synthetic Data for Text Localisation in Natural Images Code	Apr 2016
13	USM (COCO TS + ICDAR–2013)	80.4	Unsharp Masking Layer: Injecting Prior Knowledge in Convolutional Networks for Image ClassificationCode	Sep 2019
14	Neumann et al. *	77.1	Efficient Scene Text Localization and Recognition with Local Character Refinement	Apr 2015
15	Jaderberg et al.	76.8	Reading Text in the Wild with Convolutional Neural Networks	Dec 2014

h-mean

#	Model	Score	Paper / Code	Date
1	CRAFT	95.2	Character Region Awareness for Text Detection Code	Apr 2019

precision

#	Model	Score	Paper / Code	Date
1	CRAFT	97.4	Character Region Awareness for Text Detection Code	Apr 2019
2	TextFuseNet (ResNeXt-101)	97.27	TextFuseNet: Scene Text Detection with Richer Fused FeaturesCode	May 2020
3	Mask TextSpotter	95	Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes Code	Jul 2018
4	SPCNET	93.8	Scene Text Detection with Supervised Pyramid Context Network Code	Nov 2018
5	WordSup (VGG16-synth-icdar)	93.34	WordSup: Exploiting Word Annotations for Character based Text Detection	Aug 2017
6	Corner Localization (multi-scale)	92	Multi-Oriented Scene Text Detection via Corner Localization and Region Segmentation Code	Feb 2018
7	Gupta et al.	92	Synthetic Data for Text Localisation in Natural Images Code	Apr 2016
8	Corner-based Region Proposals	91.9	Detecting Multi-Oriented Text with Corner-based Region Proposals Code	Apr 2018
9	TextBoxes++_MS	91	TextBoxes++: A Single-Shot Oriented Scene Text Detector Code	Jan 2018
10	PixelLink+VGG16 2s MS	88.6	PixelLink: Detecting Scene Text via Instance Segmentation Code	Jan 2018
11	Jaderberg et al.	88.5	Reading Text in the Wild with Convolutional Neural Networks	Dec 2014
12	SSTD	88	Single Shot Text Detector with Regional Attention Code	Sep 2017
13	SegLink	87.7	Detecting Oriented Text in Natural Images by Linking Segments Code	Mar 2017
14	Neumann et al. *	81.8	Efficient Scene Text Localization and Recognition with Local Character Refinement	Apr 2015

recall

#	Model	Score	Paper / Code	Date
1	CRAFT	93.1	Character Region Awareness for Text Detection Code	Apr 2019
2	TextFuseNet (ResNeXt-101)	92.09	TextFuseNet: Scene Text Detection with Richer Fused FeaturesCode	May 2020
3	SPCNET	90.5	Scene Text Detection with Supervised Pyramid Context Network Code	Nov 2018
4	Mask TextSpotter	88.6	Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes Code	Jul 2018
5	WordSup (VGG16-synth-icdar)	87.53	WordSup: Exploiting Word Annotations for Character based Text Detection	Aug 2017
6	PixelLink+VGG16 2s MS	87.5	PixelLink: Detecting Scene Text via Instance Segmentation Code	Jan 2018
7	SSTD	86	Single Shot Text Detector with Regional Attention Code	Sep 2017
8	Corner Localization (multi-scale)	84.4	Multi-Oriented Scene Text Detection via Corner Localization and Region Segmentation Code	Feb 2018
9	TextBoxes++_MS	84	TextBoxes++: A Single-Shot Oriented Scene Text Detector Code	Jan 2018
10	Corner-based Region Proposals	83.9	Detecting Multi-Oriented Text with Corner-based Region Proposals Code	Apr 2018
11	SegLink	83	Detecting Oriented Text in Natural Images by Linking Segments Code	Mar 2017
12	Gupta et al.	75.5	Synthetic Data for Text Localisation in Natural Images Code	Apr 2016
13	Neumann et al. *	72.4	Efficient Scene Text Localization and Recognition with Local Character Refinement	Apr 2015
14	Jaderberg et al.	67.8	Reading Text in the Wild with Convolutional Neural Networks	Dec 2014