Benchmarks

327 benchmarks across 83 tasks in 15 research areas.

76Active
2509Results
83Tasks
15Areas

Top by area

Ranked by results, recency, and community interest.

Computer Vision27 tasks · 21/199
1ICDAR 2015 Incidental Scene TextScene Text Detection2Total-TextScene Text Detection3publaynet-valDocument Layout Analysis
27 tasks, 199 benchmarks →
Computer Code5 tasks · 6/13
1SWE-bench Verified SubsetCode Generation2HumanEval: Hand-Written Evaluation SetCode Generation3LiveCodeBenchCode Generation
5 tasks, 13 benchmarks →
Reasoning5 tasks · 9/19
1Mathematics Aptitude Test of HeuristicsMathematical Reasoning2Massive Multitask Language UnderstandingCommonsense Reasoning3Graduate-Level Google-Proof Q&AMulti-step Reasoning
5 tasks, 19 benchmarks →
Time Series3 tasks · 6/8
1M4 Forecasting CompetitionTime Series Forecasting2Weather Time Series BenchmarkTime Series Forecasting3Electricity Transformer Temperature - 15-minute (ETTm1)Time Series Forecasting
3 tasks, 8 benchmarks →
Agentic AI5 tasks · 6/6
1SWE-bench Verified — Agentic LeaderboardSWE-bench2WebArena: A Realistic Web Environment for Building Autonomous AgentsWeb & Desktop Agents3Human-Calibrated Autonomy Software TasksHCAST
5 tasks, 6 benchmarks →
Medical2 tasks · 5/13
1Autism Brain Imaging Data Exchange IDisease Classification2Synapse Multi-Organ Abdominal CT SegmentationMedical Image Segmentation3Brain Tumor Segmentation Challenge 2023Medical Image Segmentation
2 tasks, 13 benchmarks →
Multimodal10 tasks · 9/23
1Massive Multidiscipline Multimodal UnderstandingVisual Question Answering2TextVQA: Towards VQA Models That Can ReadVisual Question Answering3MMBench: Is Your Multi-modal Model an All-around Player?Visual Question Answering
10 tasks, 23 benchmarks →
Natural Language Processing13 tasks · 4/17
1Stanford Question Answering Dataset v2.0Question Answering2CNN/DailyMail SummarizationText Summarization3Stanford Natural Language InferenceNatural Language Inference
13 tasks, 17 benchmarks →
Speech2 tasks · 4/6
1LibriSpeech ASR CorpusSpeech Recognition2CSTR VCTK CorpusText-to-Speech3The LJ Speech DatasetText-to-Speech
2 tasks, 6 benchmarks →
Mobile Development1 tasks · 1/1
1Callstack Incubator React Native Evaluation SuiteReact Native Code Generation
Industrial Inspection1 tasks · 1/7
1MVTec Anomaly Detection DatasetAnomaly Detection2Kolektor Surface Defect Dataset 2Anomaly Detection3MVTec 3D Anomaly Detection DatasetAnomaly Detection
1 tasks, 7 benchmarks →
Reinforcement Learning2 tasks · 2/2
1Arcade Learning Environment (Atari 2600)Atari Games2Multi-Joint dynamics with ContactContinuous Control
Graphs2 tasks · 1/3
1Cora Citation NetworkNode Classification2Open Graph BenchmarkNode Classification3OGB (Open Graph Benchmark)Graph Classification
Audio4 tasks · 1/8
1AudioSetAudio Classification2Environmental Sound Classification 50Audio Classification3DIHARDVoice Activity Detection
4 tasks, 8 benchmarks →
Robots1 tasks · 0/2
1SIMPLERRobotics2RLBenchRobotics
OpenML-CC18Needs Research·Tabular Classification
5 resultsaccuracyEst. 2019Latest: Jun 2025
WebArena: A Realistic Web Environment for Building Autonomous AgentsActive·Web & Desktop Agents
6 resultssuccess-rateEst. 2023Latest: Apr 2025
Human-Calibrated Autonomy Software TasksActive·HCAST
6 resultssuccess-rateEst. 2024Latest: Apr 2025
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer EnvironmentsActive·Web & Desktop Agents
5 resultssuccess-rateEst. 2024Latest: Apr 2025
Research Engineering BenchmarkActive·RE-Bench
5 resultsnormalized-scoreEst. 2024Latest: Apr 2025
METR Autonomy Evaluation: Time HorizonActive·Time Horizon
5 resultstask-horizon-minutesEst. 2024Latest: Apr 2025
MVTec 3D Anomaly Detection DatasetNeeds Research·Anomaly Detection
6 resultsaurocEst. 2021Latest: Mar 2025
SWE-bench Verified — Agentic LeaderboardActive·SWE-bench
15 resultsresolve-rateEst. 2024Latest: Feb 2025
Massive Multidiscipline Multimodal UnderstandingActive·Visual Question Answering
11 resultsaccuracyEst. 2024Latest: Feb 2025
TextVQA: Towards VQA Models That Can ReadActive·Visual Question Answering
9 resultsaccuracyEst. 2019Latest: Feb 2025
MMBench: Is Your Multi-modal Model an All-around Player?Active·Visual Question Answering
8 resultsaccuracyEst. 2023Latest: Feb 2025
videodb's-ocr-benchmark-public-collectionNeeds Research·Optical Character Recognition
15 resultsaccuracyEst. 2020Latest: Feb 2025
Weather Time Series BenchmarkActive·Time Series Forecasting
12 resultsmseEst. 2021Latest: Feb 2025
Electricity Transformer Temperature - 15-minute (ETTm1)Active·Time Series Forecasting
6 resultsmseEst. 2021Latest: Feb 2025
Electricity Transformer Temperature - hourly (ETTh1)Active·Time Series Forecasting
6 resultsmseEst. 2021Latest: Feb 2025

All benchmarks

Optical Character Recognition114
scut-ctw1500Active
82 resultsaccuracyEst. 2020Latest: Dec 2024
cnn-/-daily-mailSaturated
80 resultsaccuracyEst. 2020Latest: May 2023
icdar2013Legacy
39 resultsaccuracyEst. 2020Latest: Aug 2023
dartNeeds Research
32 resultsaccuracyEst. 2020Latest: Oct 2023
tabfactNeeds Research
23 resultsaccuracyEst. 2020Latest: Dec 2024
icdar2015Needs Research
26 resultsaccuracyEst. 2020Latest: Aug 2023
inverse-textNeeds Research
18 resultsaccuracyEst. 2020Latest: May 2023
videodb's-ocr-benchmark-public-collectionNeeds Research
15 resultsaccuracyEst. 2020Latest: Feb 2025
sun-rgb-dNeeds Research
19 resultsaccuracyEst. 2020Latest: Jun 2021
CodeSearchNetNeeds Research
14 resultsaccuracyEst. 2020Latest: Sep 2024
pendigitsNeeds Research
15 resultsaccuracyEst. 2020Latest: May 2021
lam(line-level)Needs Research
12 resultsaccuracyEst. 2020Latest: Sep 2024
read2016(line-level)Needs Research
9 resultsaccuracyEst. 2020Latest: Sep 2024
iam(line-level)Needs Research
9 resultsaccuracyEst. 2020Latest: Sep 2024
howsumm-stepNeeds Research
11 resultsaccuracyEst. 2020Latest: Oct 2021
e2eNeeds Research
10 resultsaccuracyEst. 2020Latest: Jul 2021
howsumm-methodNeeds Research
9 resultsaccuracyEst. 2020Latest: Oct 2021
urdudocNeeds Research
9 resultsaccuracyEst. 2020Latest: Jun 2023
KITAB Arabic OCR BenchmarkNeeds Research
8 resultscerEst. 2024
belfortNeeds Research
8 resultsaccuracyEst. 2020Latest: Jun 2023
wikibioNeeds Research
8 resultsaccuracyEst. 2020Latest: Feb 2021
codesearchnet---phpNeeds Research
8 resultsaccuracyEst. 2020Latest: Apr 2021
codesearchnet---javascriptNeeds Research
8 resultsaccuracyEst. 2020Latest: Apr 2021
codesearchnet---javaNeeds Research
8 resultsaccuracyEst. 2020Latest: Apr 2021
reuters-21578Needs Research
8 resultsaccuracyEst. 2020Latest: Mar 2020
read-2016Needs Research
4 resultsaccuracyEst. 2020Latest: Sep 2024
CodeSOTA VerificationActive
benchmarking-chinese-text-recognition:-datasets,-bNeeds Research
7 resultsaccuracyEst. 2020Latest: Aug 2023
codesearchnet---pythonNeeds Research
7 resultsaccuracyEst. 2020Latest: Apr 2021
codesearchnet---rubyNeeds Research
7 resultsaccuracyEst. 2020Latest: Apr 2021
codesearchnet---goNeeds Research
7 resultsaccuracyEst. 2020Latest: Apr 2021
OCR CER BenchmarkActive
OCR WER BenchmarkActive
Internal Mistral BenchmarkActive
webnlg-(all)Needs Research
6 resultsaccuracyEst. 2020Latest: Jul 2021
webnlg-(seen)Needs Research
6 resultsaccuracyEst. 2020Latest: Jul 2021
webnlg-(unseen)Needs Research
6 resultsaccuracyEst. 2020Latest: Jul 2021
mldoc-zero-shot-english-to-frenchNeeds Research
6 resultsaccuracyEst. 2020Latest: Sep 2019
tobacco-small-3482Needs Research
6 resultsaccuracyEst. 2020Latest: Apr 2020
hocNeeds Research
6 resultsaccuracyEst. 2020Latest: Oct 2022
mldoc-zero-shot-english-to-spanishNeeds Research
6 resultsaccuracyEst. 2020Latest: Sep 2019
Thai OCR BenchmarkNeeds Research
5 resultsted-scoreEst. 2024
mldoc-zero-shot-english-to-germanNeeds Research
5 resultsaccuracyEst. 2020Latest: Sep 2019
wikipedia-person-and-animal-datasetNeeds Research
5 resultsaccuracyEst. 2020Latest: Feb 2020
mldoc-zero-shot-english-to-russianNeeds Research
5 resultsaccuracyEst. 2020Latest: Sep 2019
mldoc-zero-shot-english-to-chineseNeeds Research
5 resultsaccuracyEst. 2020Latest: Sep 2019
re-docredNeeds Research
1 resultsaccuracyEst. 2020Latest: Dec 2024
bc8Needs Research
1 resultsaccuracyEst. 2020Latest: Jan 2025
dwieNeeds Research
1 resultsaccuracyEst. 2020Latest: Dec 2024
hyperpartisan-news-detectionNeeds Research
1 resultsaccuracyEst. 2020Latest: Oct 2024
docred-ieNeeds Research
1 resultsaccuracyEst. 2020Latest: Apr 2024
lunNeeds Research
1 resultsaccuracyEst. 2020Latest: Oct 2024
stdwNeeds Research
4 resultsaccuracyEst. 2020Latest: Aug 2022
mldoc-zero-shot-english-to-italianNeeds Research
4 resultsaccuracyEst. 2020Latest: Sep 2019
bbcsportNeeds Research
4 resultsaccuracyEst. 2020Latest: Mar 2020
twitterNeeds Research
3 resultsaccuracyEst. 2020Latest: Mar 2020
cub-200-2011Needs Research
3 resultsaccuracyEst. 2020Latest: Dec 2023
rotowireNeeds Research
3 resultsaccuracyEst. 2020Latest: Aug 2021
mldoc-zero-shot-english-to-japaneseNeeds Research
3 resultsaccuracyEst. 2020Latest: Sep 2019
dareczechNeeds Research
3 resultsaccuracyEst. 2020Latest: Dec 2021
reuters-rcv1/rcv2-german-to-englishNeeds Research
3 resultsaccuracyEst. 2020Latest: Dec 2014
sutNeeds Research
3 resultsaccuracyEst. 2020Latest: Nov 2023
bbc-xsumNeeds Research
3 resultsaccuracyEst. 2020Latest: Jul 2020
fsns---testNeeds Research
3 resultsaccuracyEst. 2020Latest: Dec 2017
amazonNeeds Research
3 resultsaccuracyEst. 2020Latest: Mar 2020
reuters-rcv1/rcv2-english-to-germanNeeds Research
3 resultsaccuracyEst. 2020Latest: Dec 2014
recipeNeeds Research
2 resultsaccuracyEst. 2020Latest: Dec 2019
i2l-140kNeeds Research
2 resultsaccuracyEst. 2020Latest: Feb 2018
scidocs-(mesh)Needs Research
2 resultsaccuracyEst. 2020Latest: Feb 2022
imdb-mNeeds Research
2 resultsaccuracyEst. 2020Latest: Mar 2021
wos-5736Needs Research
2 resultsaccuracyEst. 2020Latest: Sep 2017
cedar-signatureNeeds Research
2 resultsaccuracyEst. 2020Latest: Sep 2020
clueweb09-bNeeds Research
2 resultsaccuracyEst. 2020Latest: Jun 2019
icdar-2019Needs Research
2 resultsaccuracyEst. 2020Latest: Mar 2022
textzoomNeeds Research
2 resultsaccuracyEst. 2020Latest: Nov 2022
classicNeeds Research
2 resultsaccuracyEst. 2020Latest: Dec 2019
aapdNeeds Research
2 resultsaccuracyEst. 2020Latest: Feb 2020
dise-2021-datasetNeeds Research
2 resultsaccuracyEst. 2020Latest: Oct 2022
simaraNeeds Research
2 resultsaccuracyEst. 2020Latest: Apr 2023
scidocs-(mag)Needs Research
2 resultsaccuracyEst. 2020Latest: Feb 2022
warppie10pNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2020
australianNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2020
textsegNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2022
saint-gallNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2021
scene-text-recognition-benchmarksNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2022
reuters-de-enNeeds Research
1 resultsaccuracyEst. 2020Latest: Oct 2014
food-101Needs Research
1 resultsaccuracyEst. 2020Latest: Dec 2020
hkrNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2021
wos-11967Needs Research
1 resultsaccuracyEst. 2020Latest: Sep 2017
benthamNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2021
pixraw10pNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2020
im2latex-100kNeeds Research
1 resultsaccuracyEst. 2020Latest: Feb 2018
mpqaNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2019
reuters-en-deNeeds Research
1 resultsaccuracyEst. 2020Latest: Oct 2014
yelp-14Needs Research
1 resultsaccuracyEst. 2020Latest: Apr 2019
and-datasetNeeds Research
1 resultsaccuracyEst. 2020Latest: Sep 2020
arxiv-hep-th-citation-graphNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2021
irisNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2020
wos-46985Needs Research
1 resultsaccuracyEst. 2020Latest: Sep 2017
ephoieNeeds Research
1 resultsaccuracyEst. 2020Latest: Apr 2022
iam-bNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2021
cl-scisummNeeds Research
1 resultsaccuracyEst. 2020Latest: Sep 2019
iam-dNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2021
wikilingua-(tr->en)Needs Research
1 resultsaccuracyEst. 2020Latest: Dec 2021
mldoc-zero-shot-german-to-frenchNeeds Research
1 resultsaccuracyEst. 2020Latest: May 2018
digital-peterNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2021
baNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2020
arxiv-summarization-datasetNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2021
wineNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2020
jaffeNeeds Research
1 resultsaccuracyEst. 2020Latest: Nov 2020
CodeSOTA Polish OCR BenchmarkNeeds Research
cerEst. 2025
IMPACT Polish Digital Libraries Ground TruthNeeds Research
cerEst. 2012
PolEval 2021 OCR Post-Correction TaskNeeds Research
cerEst. 2021
Scanned Receipts OCR and Information ExtractionNeeds Research
f1Est. 2019
Scene Text Detection11
ICDAR 2015 Incidental Scene TextActive
188 resultsf1Est. 2015Latest: Apr 2023
Total-TextNeeds Research
108 resultsf1Est. 2017Latest: Aug 2023
msra-td500Needs Research
61 resultsaccuracyEst. 2020Latest: Aug 2023
icdar-2013Legacy
49 resultsaccuracyEst. 2020Latest: Jul 2022
icdar-2017-mltNeeds Research
42 resultsaccuracyEst. 2020Latest: Dec 2019
coco-textNeeds Research
33 resultsaccuracyEst. 2020Latest: May 2023
Curved Text in the Wild 1500Needs Research
18 resultsf1Est. 2019Latest: Feb 2022
Union14M: A Unified Benchmark for Scene Text RecognitionActive
8 resultsaccuracyEst. 2023
ic19-artNeeds Research
8 resultsaccuracyEst. 2020Latest: Aug 2023
ICDAR 2019 Arbitrary-Shaped TextNeeds Research
4 resultsf1Est. 2019Latest: Sep 2019
ic19-rectsNeeds Research
1 resultsaccuracyEst. 2020Latest: Jun 2019
Document Layout Analysis5
publaynet-valActive
85 resultsaccuracyEst. 2020Latest: Dec 2024
document-layout-recognition-challenge-testNeeds Research
18 resultsaccuracyEst. 2020
document-layout-recognition-challenge-mini-devNeeds Research
12 resultsaccuracyEst. 2020
u-diads-bibNeeds Research
8 resultsaccuracyEst. 2020Latest: Sep 2024
d4laNeeds Research
3 resultsaccuracyEst. 2020Latest: Dec 2024
Scene Text Recognition11
svtActive
40 resultsaccuracyEst. 2020Latest: Aug 2023
iiit5kNeeds Research
21 resultsaccuracyEst. 2020Latest: Aug 2023
cute80Needs Research
20 resultsaccuracyEst. 2020Latest: Aug 2023
svtpNeeds Research
19 resultsaccuracyEst. 2020Latest: Aug 2023
icdar-2003Needs Research
12 resultsaccuracyEst. 2020Latest: Mar 2022
wostNeeds Research
5 resultsaccuracyEst. 2020Latest: May 2023
uber-textNeeds Research
3 resultsaccuracyEst. 2020Latest: May 2023
hostNeeds Research
3 resultsaccuracyEst. 2020Latest: May 2023
msdaNeeds Research
2 resultsaccuracyEst. 2020Latest: Aug 2021
ic13Needs Research
1 resultsaccuracyEst. 2020Latest: May 2023
svt-pNeeds Research
1 resultsaccuracyEst. 2020Latest: May 2023
Document Parsing2
OmniDocBench v1.5Needs Research
28 resultscompositeEst. 2024
olmOCR-BenchActive
28 resultspass-rateEst. 2024
Document Image Classification7
rvl-cdipActive
33 resultsaccuracyEst. 2020Latest: Dec 2024
tobacco-3482Needs Research
14 resultsaccuracyEst. 2020Latest: Jan 2023
noisy-bangla-numeralNeeds Research
2 resultsaccuracyEst. 2020Latest: Aug 2019
noisy-bangla-charactersNeeds Research
2 resultsaccuracyEst. 2020Latest: Aug 2019
noisy-mnistNeeds Research
1 resultsaccuracyEst. 2020Latest: Aug 2019
n-mnistNeeds Research
1 resultsaccuracyEst. 2020Latest: Jun 2018
aipNeeds Research
1 resultsaccuracyEst. 2020Latest: Mar 2021
Object Detection3object-detection
Microsoft COCO: Common Objects in ContextActive
13 resultsmAPEst. 2014
Large Vocabulary Instance Segmentation v1.0Active
16 resultsmask-apEst. 2019Latest: Nov 2024
Pascal Visual Object Classes Challenge 2012Needs Research
6 resultsmAPEst. 2012Latest: Dec 2015
General OCR Capabilities4
OCRBench v2Active
32 resultsoverall-en-privateEst. 2024
Comprehensive Challenge OCRNeeds Research
12 resultsmulti-scene-f1Est. 2024
MME Video OCR BenchmarkNeeds Research
6 resultstotal-accuracyEst. 2024
reVISION Polish Vision-Language BenchmarkNeeds Research
accuracyEst. 2025
Handwriting Recognition7
IAM Handwriting DatabaseActive
22 resultscerEst. 1999Latest: Sep 2024
Cultural Heritage Understanding Research Repository OCR DatasetNeeds Research
8 resultshandwritten-levenshteinEst. 2024
RIMES French Handwriting DatabaseActive
cerEst. 2011
kohtdNeeds Research
4 resultsaccuracyEst. 2020Latest: Sep 2021
banglalekha-isolated-datasetNeeds Research
3 resultsaccuracyEst. 2020Latest: Aug 2020
an-extensive-dataset-of-handwritten-central-kurdisNeeds Research
1 resultsaccuracyEst. 2020Latest: Oct 2022
EMNIST Extended with Polish DiacriticsNeeds Research
accuracyEst. 2020
Table Recognition5
pubtabnetActive
18 resultsaccuracyEst. 2020Latest: Apr 2024
table-recognition-challenge-mini-testNeeds Research
12 resultsaccuracyEst. 2020
table-recognition-challenge-testNeeds Research
6 resultsaccuracyEst. 2020
wtwNeeds Research
1 resultsaccuracyEst. 2020Latest: Mar 2023
icdar2013-table-structure-recognitionLegacy
1 resultsaccuracyEst. 2020Latest: Apr 2021
Image Classification4image-classification
ImageNet Large Scale Visual Recognition Challenge 2012Active
16 resultstop-1-accuracyEst. 2012
Canadian Institute for Advanced Research 100Needs Research
4 resultsaccuracyEst. 2009
Canadian Institute for Advanced Research 10Needs Research
3 resultsaccuracyEst. 2009
ImageNet-V2 Matched FrequencyNeeds Research
2 resultstop-1-accuracyEst. 2019
Document Understanding2document-question-answering
DocLayNet: A Large Human-Annotated Dataset for Document-Layout AnalysisActive
7 resultsmAPEst. 2022
Form Understanding in Noisy Scanned DocumentsActive
f1Est. 2019
Semantic Segmentation2image-segmentation
ADE20K Scene Parsing BenchmarkActive
6 resultsmIoUEst. 2016
Cityscapes DatasetNeeds Research
mIoUEst. 2016
Unconditional Image Generation2unconditional-image-generation
LSUN Bedroom FIDNeeds Research
fidEst. 2015
CIFAR-10 FIDNeeds Research
fidEst. 2009
Depth Estimation2depth-estimation
NYU Depth V2Needs Research
abs-relEst. 2012
KITTI DepthNeeds Research
abs-relEst. 2012
Text-to-Video2text-to-video
EvalCrafterNeeds Research
compositeEst. 2023
VBenchNeeds Research
compositeEst. 2023
Zero-Shot Image Classification1zero-shot-image-classification
ImageNet Zero-ShotNeeds Research
top-1-accuracyEst. 2009
Video Classification3video-classification
UCF-101Needs Research
top-1-accuracyEst. 2012
Kinetics-400Needs Research
top-1-accuracyEst. 2017
Something-Something V2Needs Research
top-1-accuracyEst. 2017
Image-to-Image2image-to-image
Urban100Needs Research
psnrEst. 2015
Set5Needs Research
psnrEst. 2012
Zero-Shot Object Detection2zero-shot-object-detection
OmniLabelNeeds Research
mapEst. 2023
LVIS Zero-ShotNeeds Research
mapEst. 2019
Video-to-Video1video-to-video
DAVISNeeds Research
j-and-fEst. 2016
Keypoint Detection2keypoint-detection
COCO KeypointsNeeds Research
mapEst. 2014
MPII Human PoseNeeds Research
accuracyEst. 2014
Image-to-Video1image-to-video
I2VBenchNeeds Research
compositeEst. 2024
Image-to-3D1image-to-3d
GSO (Google Scanned Objects)Needs Research
compositeEst. 2022
Image Feature Extraction1image-feature-extraction
ImageNet Linear ProbeNeeds Research
top-1-accuracyEst. 2009
Text-to-3D1text-to-3d
T3BenchNeeds Research
compositeEst. 2023
Mask Generation1mask-generation
SA-1BNeeds Research
iouEst. 2023
Mathematical Reasoning4
Mathematics Aptitude Test of HeuristicsNeeds Research
34 resultsaccuracyEst. 2021
Grade School Math 8KActive
15 resultsaccuracyEst. 2021
American Invitational Mathematics Examination 2025Active
5 resultsaccuracyEst. 2025
American Invitational Mathematics Examination 2024Needs Research
8 resultsaccuracyEst. 2024
Commonsense Reasoning5
Massive Multitask Language UnderstandingActive
24 resultsaccuracyEst. 2021
AI2 Reasoning ChallengeNeeds Research
10 resultsaccuracyEst. 2018
HellaSwagNeeds Research
5 resultsaccuracyEst. 2019
CommonsenseQANeeds Research
3 resultsaccuracyEst. 2019
WinoGrandeNeeds Research
3 resultsaccuracyEst. 2019
Multi-step Reasoning4
Graduate-Level Google-Proof Q&AActive
24 resultsaccuracyEst. 2024
BIG-Bench Hard (BBH)Active
5 resultsaccuracyEst. 2022
StrategyQANeeds Research
2 resultsaccuracyEst. 2021
HotpotQANeeds Research
2 resultsf1Est. 2018
Logical Reasoning4
Abstraction and Reasoning Corpus for AGI (v1)Active
5 resultsaccuracyEst. 2024
Abstraction and Reasoning Corpus for AGI (v2)Active
3 resultsaccuracyEst. 2025
LogiQAActive
2 resultsaccuracyEst. 2020
Reading Comprehension Dataset Requiring Logical ReasoningNeeds Research
2 resultsaccuracyEst. 2020
Arithmetic Reasoning2
Math Word Problem RepositoryActive
3 resultsaccuracyEst. 2016
Simple Variations on Arithmetic Math Word ProblemsNeeds Research
3 resultsaccuracyEst. 2021
Code Generation9
SWE-bench Verified SubsetNeeds Research
38 resultsresolve-rateEst. 2024
HumanEval: Hand-Written Evaluation SetActive
30 resultspass@1Est. 2021Latest: Sep 2024
LiveCodeBenchActive
25 resultspass@1Est. 2024Latest: Mar 2024
Mostly Basic Python ProblemsNeeds Research
19 resultspass@1Est. 2021Latest: Sep 2024
SWE-bench: Software Engineering BenchmarkNeeds Research
resolve-rateEst. 2023
MBPP+ Extended VersionNeeds Research
pass@1Est. 2023
HumanEval+ Extended VersionNeeds Research
pass@1Est. 2023
Automated Programming Progress StandardNeeds Research
pass@1Est. 2021
CodeContests Competitive ProgrammingNeeds Research
pass@1Est. 2022
Code Translation1
TransCoder Evaluation on GeeksForGeeks Algorithmic ProblemsActive
7 resultscomputational-accuracyEst. 2020Latest: Sep 2024
Code Completion1
Cross-File Code Completion EvaluationActive
6 resultsexact-matchEst. 2023Latest: Sep 2024
Bug Detection1
Bugs2Fix: Learning to Rewrite Buggy CodeActive
6 resultsaccuracyEst. 2019Latest: Sep 2024
Program Repair1
Defects4J: A Database of Real Faults in Java ProgramsActive
5 resultscorrect-patchesEst. 2014Latest: Apr 2024
Time Series Forecasting6time-series-forecasting
M4 Forecasting CompetitionActive
39 resultssmapiEst. 2018Latest: Dec 2024
Weather Time Series BenchmarkActive
12 resultsmseEst. 2021Latest: Feb 2025
Electricity Transformer Temperature - 15-minute (ETTm1)Active
6 resultsmseEst. 2021Latest: Feb 2025
Electricity Transformer Temperature - hourly (ETTh1)Active
6 resultsmseEst. 2021Latest: Feb 2025
Electricity Transformer Temperature - 15-minute 2 (ETTm2)Active
6 resultsmseEst. 2021Latest: Feb 2025
Electricity Transformer Temperature - hourly 2 (ETTh2)Active
6 resultsmseEst. 2021Latest: Feb 2025
Tabular Classification1tabular-classification
OpenML-CC18Needs Research
5 resultsaccuracyEst. 2019Latest: Jun 2025
Tabular Regression1tabular-regression
California HousingNeeds Research
2 resultsrmseEst. 1997
Text Classification2text-classification
General Language Understanding EvaluationActive
7 resultsaverage-scoreEst. 2018Latest: Jul 2024
SuperGLUENeeds Research
7 resultsaverage-scoreEst. 2019Latest: Jul 2024
Question Answering1question-answering
Stanford Question Answering Dataset v2.0Active
9 resultsf1Est. 2018Latest: Jul 2024
Text Summarization1summarization
CNN/DailyMail SummarizationSaturated
15 resultsrouge-1Est. 2015Latest: Jul 2024
Natural Language Inference1
Stanford Natural Language InferenceActive
8 resultsaccuracyEst. 2015Latest: Jul 2024
Named Entity Recognition1token-classification
CoNLL-2003 Named Entity RecognitionActive
7 resultsf1Est. 2003Latest: Jul 2024
Text Ranking2text-ranking
BEIRNeeds Research
4 resultsndcgEst. 2021Latest: Sep 2024
MS MARCONeeds Research
4 resultsmrrEst. 2016Latest: Oct 2023
Feature Extraction1feature-extraction
MTEB LeaderboardNeeds Research
6 resultsaccuracyEst. 2022Latest: Sep 2024
Machine Translation2translation
WMT'23Needs Research
4 resultsbleuEst. 2023
FLORES-200Needs Research
bleuEst. 2022
Zero-Shot Classification1zero-shot-classification
XNLINeeds Research
3 resultsaccuracyEst. 2018Latest: Jan 2023
Semantic Textual Similarity1sentence-similarity
STS BenchmarkNeeds Research
3 resultsspearmanEst. 2017Latest: Jan 2024
Table Question Answering2table-question-answering
WikiTableQuestionsNeeds Research
3 resultsaccuracyEst. 2015Latest: Apr 2020
SQANeeds Research
accuracyEst. 2017
Fill-Mask1fill-mask
GLUENeeds Research
3 resultsaccuracyEst. 2018Latest: Jan 2023
Language Modeling1text-generation
WikiText PerplexityNeeds Research
perplexityEst. 2016
Web & Desktop Agents2
WebArena: A Realistic Web Environment for Building Autonomous AgentsActive
6 resultssuccess-rateEst. 2023Latest: Apr 2025
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer EnvironmentsActive
5 resultssuccess-rateEst. 2024Latest: Apr 2025
SWE-bench1
SWE-bench Verified — Agentic LeaderboardActive
15 resultsresolve-rateEst. 2024Latest: Feb 2025
HCAST1
Human-Calibrated Autonomy Software TasksActive
6 resultssuccess-rateEst. 2024Latest: Apr 2025
RE-Bench1
Research Engineering BenchmarkActive
5 resultsnormalized-scoreEst. 2024Latest: Apr 2025
Time Horizon1
METR Autonomy Evaluation: Time HorizonActive
5 resultstask-horizon-minutesEst. 2024Latest: Apr 2025
Medical Image Segmentation4
Synapse Multi-Organ Abdominal CT SegmentationActive
11 resultsmean-dscEst. 2015Latest: Jan 2024
Brain Tumor Segmentation Challenge 2023Active
3 resultsmean-dice-wt-tc-etEst. 2023Latest: Jun 2024
Automated Cardiac Diagnosis ChallengeActive
6 resultsmean-dscEst. 2017Latest: Mar 2024
Beyond The Cranial Vault Multi-Organ CT SegmentationActive
6 resultsmean-dscEst. 2015Latest: Aug 2023
Disease Classification9
Autism Brain Imaging Data Exchange IActive
21 resultsaccuracyEst. 2012
CheXpert: A Large Chest Radiograph Dataset with Uncertainty LabelsNeeds Research
7 resultsaurocEst. 2019
NIH Clinical Center Chest X-ray DatasetNeeds Research
4 resultsaurocEst. 2017
RSNA Pneumonia Detection ChallengeNeeds Research
3 resultsmapEst. 2018Latest: Jan 2024
MIMIC-CXR: Medical Information Mart for Intensive Care - Chest X-rayNeeds Research
3 resultsaurocEst. 2019
VinDr-CXR: Vietnamese Dataset for Chest RadiographNeeds Research
2 resultsaurocEst. 2022
Autism Brain Imaging Data Exchange IINeeds Research
2 resultsaccuracyEst. 2017
COVID-19 Image Data CollectionNeeds Research
2 resultsaurocEst. 2020
PadChest: A Large Chest X-ray Image DatasetNeeds Research
1 resultsaurocEst. 2020
Visual Question Answering6visual-question-answering
Massive Multidiscipline Multimodal UnderstandingActive
11 resultsaccuracyEst. 2024Latest: Feb 2025
TextVQA: Towards VQA Models That Can ReadActive
9 resultsaccuracyEst. 2019Latest: Feb 2025
MMBench: Is Your Multi-modal Model an All-around Player?Active
8 resultsaccuracyEst. 2023Latest: Feb 2025
Visual Question Answering v2.0Active
7 resultsaccuracyEst. 2017Latest: Oct 2024
Outside Knowledge Visual Question AnsweringActive
accuracyEst. 2019
GQA: Visual Reasoning in the Real WorldActive
accuracyEst. 2019
Image Captioning2image-to-text
COCO CaptionsActive
2 resultsciderEst. 2015Latest: Jan 2023
Novel Object Captioning at ScaleActive
ciderEst. 2019
Audio-Text-to-Text2audio-text-to-text
VoiceBench: Benchmarking LLM-Based Voice AssistantsActive
accuracyEst. 2024
AudioBenchNeeds Research
accuracyEst. 2024
Any-to-Any1any-to-any
DEMON BenchNeeds Research
accuracyEst. 2024
Text-to-Image Generation3text-to-image
DPG-BenchNeeds Research
compositeEst. 2024
MJHQ-30K FIDNeeds Research
fidEst. 2024
GenEvalNeeds Research
accuracyEst. 2023
Cross-Modal Retrieval1visual-document-retrieval
ViDoReNeeds Research
ndcg-at-5Est. 2024
Image-Text-to-Image2image-text-to-image
InstructPix2PixNeeds Research
clip-scoreEst. 2023
MagicBrushNeeds Research
clip-scoreEst. 2023
Image-Text-to-Text3image-text-to-text
MMMUNeeds Research
accuracyEst. 2023
MMStarNeeds Research
accuracyEst. 2024
MMBenchNeeds Research
accuracyEst. 2023
Video Understanding2video-text-to-text
Video-MMENeeds Research
accuracyEst. 2024
MVBenchNeeds Research
accuracyEst. 2024
Image-Text-to-Video1image-text-to-video
VideoBenchNeeds Research
compositeEst. 2024
Speech Recognition4automatic-speech-recognition
LibriSpeech ASR CorpusActive
17 resultswerEst. 2015Latest: Apr 2024
FLEURS: Few-shot Learning Evaluation of Universal Representations of SpeechActive
werEst. 2022
WildASR: A Multilingual Diagnostic Benchmark for ASR RobustnessActive
werEst. 2025
Mozilla Common VoiceNeeds Research
3 resultswerEst. 2019Latest: Dec 2022
Text-to-Speech2text-to-speech
CSTR VCTK CorpusActive
6 resultsmosEst. 2019Latest: Jun 2024
The LJ Speech DatasetNeeds Research
5 resultsmosEst. 2017Latest: Jun 2024
Anomaly Detection7
MVTec Anomaly Detection DatasetActive
9 resultsaurocEst. 2019Latest: Aug 2023
Kolektor Surface Defect Dataset 2Needs Research
6 resultsaurocEst. 2021Latest: Aug 2024
MVTec 3D Anomaly Detection DatasetNeeds Research
6 resultsaurocEst. 2021Latest: Mar 2025
X-Ray Weld Defect Detection DatasetNeeds Research
1 resultsmapEst. 2021
Visual Anomaly DatasetNeeds Research
3 resultsaurocEst. 2022
NEU Surface Defect DatabaseNeeds Research
1 resultsmapEst. 2013
Severstal Steel Defect DetectionNeeds Research
1 resultsdiceEst. 2019
React Native Code Generation1
Callstack Incubator React Native Evaluation SuiteActive
40 resultsrequirement-satisfactionEst. 2025
Atari Games1reinforcement-learning
Arcade Learning Environment (Atari 2600)Active
9 resultshuman-normalized-scoreEst. 2013
Continuous Control1
Multi-Joint dynamics with ContactActive
9 resultsaverage-returnEst. 2012
Node Classification2graph-ml
Cora Citation NetworkActive
6 resultsaccuracyEst. 2000Latest: Apr 2019
Open Graph BenchmarkNeeds Research
accuracyEst. 2020
Graph Classification1graph-ml
OGB (Open Graph Benchmark)Needs Research
accuracyEst. 2020
Audio Classification2audio-classification
AudioSetActive
mapEst. 2017
Environmental Sound Classification 50Needs Research
accuracyEst. 2015
Voice Activity Detection2voice-activity-detection
DIHARDNeeds Research
derEst. 2018
AVA-SpeechNeeds Research
accuracyEst. 2018
Audio-to-Audio2audio-to-audio
VCTK (Voice Conversion)Needs Research
pesqEst. 2019
DNS ChallengeNeeds Research
si-snrEst. 2020
Text-to-Audio2text-to-audio
AudioCapsNeeds Research
fadEst. 2019
MusicCapsNeeds Research
fadEst. 2023
Robotics2robotics
SIMPLERNeeds Research
success-rateEst. 2024
RLBenchNeeds Research
success-rateEst. 2020