Help Prioritize Research

188 benchmarks need research to determine if they're still relevant. Vote on which ones should be updated first.

188 Undecided
View Saturated/Legacy

How this works

1

Vote for benchmarks

Click the upvote button on benchmarks you want us to research

2

We research top voted

Using Exa and manual review to find latest SOTA results

3

Benchmarks get updated

Fresh data gets added, or benchmark marked as saturated/legacy

Top Voted

OmniDocBench v1.5Computer Vision/Document Parsing
23 resultsNo paper dates

Computer Vision152

OmniDocBench v1.5/Document Parsing
23 resultsNo paper dates
Total-Text/Scene Text Detection
108 resultsLatest: Aug 2023
msra-td500/Scene Text Detection
61 resultsLatest: Aug 2023
icdar-2017-mlt/Scene Text Detection
42 resultsLatest: Dec 2019
dart/Optical Character Recognition
32 resultsLatest: Oct 2023
icdar2015/Optical Character Recognition
26 resultsLatest: Aug 2023
tabfact/Optical Character Recognition
23 resultsLatest: Dec 2024
coco-text/Scene Text Detection
21 resultsLatest: May 2023
sun-rgb-d/Optical Character Recognition
19 resultsLatest: Jun 2021
inverse-text/Optical Character Recognition
18 resultsLatest: May 2023
document-layout-recognition-challenge-test/Document Layout Analysis
18 resultsNo paper dates
cute80/Scene Text Recognition
16 resultsLatest: Aug 2023
iiit5k/Scene Text Recognition
16 resultsLatest: Aug 2023
svtp/Scene Text Recognition
15 resultsLatest: Aug 2023
videodb's-ocr-benchmark-public-collection/Optical Character Recognition
15 resultsLatest: Feb 2025
pendigits/Optical Character Recognition
15 resultsLatest: May 2021
tobacco-3482/Document Image Classification
14 resultsLatest: Jan 2023
Comprehensive Challenge OCR/General OCR Capabilities
12 resultsNo paper dates
lam(line-level)/Optical Character Recognition
12 resultsLatest: Sep 2024
table-recognition-challenge-mini-test/Table Recognition
12 resultsNo paper dates

+132 more...

Reasoning10

Mathematics Aptitude Test of Heuristics/Mathematical Reasoning
5 resultsNo paper dates
HellaSwag/Commonsense Reasoning
4 resultsNo paper dates
AI2 Reasoning Challenge/Commonsense Reasoning
4 resultsNo paper dates
American Invitational Mathematics Examination 2024/Mathematical Reasoning
3 resultsNo paper dates
Simple Variations on Arithmetic Math Word Problems/Arithmetic Reasoning
3 resultsNo paper dates
WinoGrande/Commonsense Reasoning
3 resultsNo paper dates
CommonsenseQA/Commonsense Reasoning
3 resultsNo paper dates
Reading Comprehension Dataset Requiring Logical Reasoning/Logical Reasoning
2 resultsNo paper dates
StrategyQA/Multi-step Reasoning
2 resultsNo paper dates
HotpotQA/Multi-step Reasoning
2 resultsNo paper dates

Medical8

CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels/Disease Classification
7 resultsNo paper dates
NIH Clinical Center Chest X-ray Dataset/Disease Classification
4 resultsNo paper dates
MIMIC-CXR: Medical Information Mart for Intensive Care - Chest X-ray/Disease Classification
3 resultsNo paper dates
VinDr-CXR: Vietnamese Dataset for Chest Radiograph/Disease Classification
2 resultsNo paper dates
RSNA Pneumonia Detection Challenge/Disease Classification
2 resultsNo paper dates
Autism Brain Imaging Data Exchange II/Disease Classification
2 resultsNo paper dates
COVID-19 Image Data Collection/Disease Classification
2 resultsNo paper dates
PadChest: A Large Chest X-ray Image Dataset/Disease Classification
1 resultsNo paper dates

Computer Code7

SWE-bench Verified Subset/Code Generation
3 resultsNo paper dates
Mostly Basic Python Problems/Code Generation
2 resultsNo paper dates
CodeContests Competitive Programming/Code Generation
0 resultsNo paper dates
HumanEval+ Extended Version/Code Generation
0 resultsNo paper dates
Automated Programming Progress Standard/Code Generation
0 resultsNo paper dates
SWE-bench: Software Engineering Benchmark/Code Generation
0 resultsNo paper dates
MBPP+ Extended Version/Code Generation
0 resultsNo paper dates

Industrial Inspection6

Visual Anomaly Dataset/Anomaly Detection
3 resultsNo paper dates
NEU Surface Defect Database/Anomaly Detection
1 resultsNo paper dates
Severstal Steel Defect Detection/Anomaly Detection
1 resultsNo paper dates
X-Ray Weld Defect Detection Dataset/Anomaly Detection
1 resultsNo paper dates
Kolektor Surface Defect Dataset 2/Anomaly Detection
0 resultsNo paper dates
MVTec 3D Anomaly Detection Dataset/Anomaly Detection
0 resultsNo paper dates

Speech2

The LJ Speech Dataset/Text-to-Speech
0 resultsNo paper dates
Mozilla Common Voice/Speech Recognition
0 resultsNo paper dates

Audio1

Environmental Sound Classification 50/Audio Classification
0 resultsNo paper dates

Natural Language Processing1

SuperGLUE/Text Classification
0 resultsNo paper dates

Graphs1

Open Graph Benchmark/Node Classification
0 resultsNo paper dates