The General Language Understanding Evaluation (GLUE, 2018) and its harder successor SuperGLUE (2019) — a multi-task NLU benchmark covering 9 sub-tasks (CoLA, SST-2, MRPC, STS-B, MNLI, BoolQ, COPA, WSC, ReCoRD). The leaderboard saturated near human baseline around 91 in 2022 and has seen no frontier submissions since; current frontier evaluation has moved to MMLU, GPQA, BIG-Bench Hard, and HELM.
Effectively retired since late 2022. Aggregate score plateaued at 91.2–91.3 (ST-MoE-32B, Vega v2); frontier LLMs (GPT-4/5, Claude, Gemini, Llama) do not submit. Human baseline: 89.8.
5 results indexed across 1 metric. Shaded row marks current SOTA; ties broken by submission date.
| # | Model | Org | Submitted | Paper / code | SuperGLUE avg |
|---|---|---|---|---|---|
| 01 | Vega v2 (6B)API | JD Explore Academy | Oct 2022 | Toward Efficient Language Model Pretraining and Downstre… | 91.30 |
| 02 | ST-MoE-32BOSS | Google Brain | Feb 2022 | ST-MoE: Designing Stable and Transferable Sparse Expert … | 91.20 |
| 03 | ERNIE 3.0API | Baidu | Jul 2021 | ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training f… | 90.60 |
| 04 | DeBERTa (ensemble)OSS | Microsoft | Jan 2021 | DeBERTa: Decoding-enhanced BERT with Disentangled Attent… | 90.30 |
| 05 | T5-11BOSS | Oct 2019 | Exploring the Limits of Transfer Learning with a Unified… | 89.30 |
Each row below marks a model that broke the previous record on SuperGLUE avg. Intermediate submissions are kept in the leaderboard above; only SOTA-setting entries are re-listed here.
Higher scores win. Each subsequent entry improved upon the previous best.
Every paper below corresponds to at least one row in the leaderboard above. Click through for the arXiv preprint and, when available, the reference implementation.
Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.