Codesota · Computer Vision · Optical Character Recognition · e2eTasks/Computer Vision/Optical Character Recognition
Optical Character Recognition · benchmark dataset · 2020 · EN

e2e.

Dataset from Papers With Code

Saturated benchmark

Benchmark abandoned or no longer evaluated by the community

Submit a result ↵
§ 01 · Leaderboard

Best published scores.

45 results indexed across 5 metrics. Shaded row marks current SOTA; ties broken by submission date.

Primary
accuracy · higher is better
All metrics
bleu, cider, meteor, nist, rouge-l
bleu
9 rows
#ModelOrgSubmittedPaper / codebleu
01GPT-2-Large (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …70.30
02HTLM (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …70.30
03HTLM (prefix-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …70.10
04GPT-2-Medium (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …69.70
05GPT-2-Large (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …68.50
06GPT-2-Medium (fine-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …68.20
07T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…66.95
08BART-base (STSM)MetaJan 2024Self-training from Self-memory in Data-to-text Generatio…65.74
09FLAN-T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…65.65
cider
9 rows
#ModelOrgSubmittedPaper / codecider
01GPT-2-Medium (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …2.49
02GPT-2-Large (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …2.47
03GPT-2-Medium (fine-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …2.47
04HTLM (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …2.47
05GPT-2-Large (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …2.45
06HTLM (prefix-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …2.45
07T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…2.27
08BART-base (STSM)MetaJan 2024Self-training from Self-memory in Data-to-text Generatio…2.20
09FLAN-T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…2.12
meteor
9 rows
#ModelOrgSubmittedPaper / codemeteor
01HTLM (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …46.30
02GPT-2-Large (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …46.20
03GPT-2-Medium (fine-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …46.20
04HTLM (prefix-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …46.10
05GPT-2-Medium (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …46.10
06GPT-2-Large (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …46
07T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…45.70
08BART-base (STSM)MetaJan 2024Self-training from Self-memory in Data-to-text Generatio…45.60
09FLAN-T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…45.54
nist
9 rows
#ModelOrgSubmittedPaper / codenist
01HTLM (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …8.90
02HTLM (prefix-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …8.85
03GPT-2-Large (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …8.85
04GPT-2-Medium (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …8.81
05GPT-2-Large (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …8.78
06GPT-2-Medium (fine-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …8.62
07T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…8.59
08FLAN-T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…8.49
09BART-base (STSM)MetaJan 2024Self-training from Self-memory in Data-to-text Generatio…8.46
rouge-l
9 rows
#ModelOrgSubmittedPaper / coderouge-l
01GPT-2-Large (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …71.70
02GPT-2-Medium (prefix-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …71.40
03HTLM (prefix-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …71.20
04GPT-2-Medium (fine-tuning)OpenAIJul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …71
05HTLM (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …70.80
06GPT-2-Large (fine-tuning)—Jul 2021HTLM: Hyper-Text Pre-Training and Prompting of Language …69.90
07T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…68.97
08BART-base (STSM)MetaJan 2024Self-training from Self-memory in Data-to-text Generatio…68.76
09FLAN-T5-base (STSM)GoogleJan 2024Self-training from Self-memory in Data-to-text Generatio…67.85
Fig 2 · Rows sorted by score within each metric. Shaded row marks SOTA. Dates reflect model or paper release where available, otherwise the date Codesota accessed the source.
§ 04 · Literature

2 papers
tied to this benchmark.

Every paper below corresponds to at least one row in the leaderboard above. Click through for the arXiv preprint and, when available, the reference implementation.

§ 06 · Contribute

Have a score that beats
this table?

Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.

Submit a result ↵Read submission guide
What a submission needs
  • 01A public checkpoint or API endpoint
  • 02A reproduction script with frozen commit + seed
  • 03Declared evaluation environment (Python, deps)
  • 04One row per metric declared by this dataset
  • 05A contact so we can follow up on discrepancies