Stanford Question Answering Dataset v2.0.

150K questions on Wikipedia articles, including 50K unanswerable questions. Tests reading comprehension and knowing when a question cannot be answered.

Best published scores.

24 results indexed across 2 metrics. Shaded row marks current SOTA; ties broken by submission date.

Primary

f1 · higher is better

All metrics

em, f1

7 steps
of state of the art.

Each row below marks a model that broke the previous record on f1. Intermediate submissions are kept in the leaderboard above; only SOTA-setting entries are re-listed here.

Higher scores win. Each subsequent entry improved upon the previous best.

1. Jun 1, 2016Logistic Regression (SQuAD baseline)Stanford NLP51
2. Jan 1, 2018SLQA+ (single model)Alibaba iDST87.02
3. Mar 1, 2019BERT + ConvLSTM + MTL + Verifier (ensemble)Layer 6 AI89.29
4. Nov 1, 2019RoBERTa+Verify (single model)CW89.59
5. May 1, 2020Enhanced Albert+Verifier3 (ensemble)Microsoft STCA AIC89.78
6. Jul 1, 2020RoBERTa (single model)Facebook AI89.80
7. Nov 18, 2021DeBERTa-v3-largeMicrosoft91.40

6 papers
tied to this benchmark.

Every paper below corresponds to at least one row in the leaderboard above. Click through for the arXiv preprint and, when available, the reference implementation.

Have a score that beats
this table?

Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.

• 01A public checkpoint or API endpoint
• 02A reproduction script with frozen commit + seed
• 03Declared evaluation environment (Python, deps)
• 04One row per metric declared by this dataset
• 05A contact so we can follow up on discrepancies