Large language models, measured honestly.

The original MMLU: 15,908 four-choice questions across 57 subjects from elementary to professional level. Largely saturated at the frontier — top models cluster above 90%. For a harder variant see MMLU-Pro.

#	Model	Vendor	accuracy
01	o3	OpenAI	92.9%
02	GPT-5.2	OpenAI	92.4%
03	Claude Opus 4.5	Anthropic	91.8%
04	o1	OpenAI	91.8%
05	Claude Opus 4.5	Anthropic	91.6%
06	Gemini 3 Pro	Google	91.4%
07	Claude Opus 4.6	Anthropic	91.2%
08	o1-preview	OpenAI	90.8%
09	GPT-5	OpenAI	90.8%
10	DeepSeek R1	DeepSeek	90.8%
11	GPT-4.5 Preview	OpenAI	90.8%
12	Claude Sonnet 4.5	Anthropic	90.4%
13	GPT-4.1	OpenAI	90.2%
14	Claude Sonnet 4	Anthropic	90.1%
15	o4-mini	OpenAI	90%
16	GLM-4.5	Zhipu AI	90%
17	Gemini 2.5 Pro	Google	89.8%
18	Gemini 3 Flash	Google	89.6%
19	Llama 4 Maverick	Meta	89.4%
20	Claude Opus 4	Anthropic	88.8%
21	Qwen 3 72B	Alibaba	88.7%
22	Llama 3.1 405B	Meta	88.6%
23	DeepSeek-V3	DeepSeek	88.5%
24	MiniMax-Text-01	MiniMax	88.5%
25	Claude 3.5 Sonnet	Anthropic	88.3%
26	DeepSeek V3.5	DeepSeek	88.2%
27	Qwen3-235B-A22B	Alibaba	87.81%
28	Llama 4 405B	Meta	87.8%
29	Qwen3-Coder-Next	Qwen	87.73%
30	Grok 2	xAI	87.5%
31	Llama 3 (405B, Instruct)	Meta	87.3%
32	Trinity Large Preview	Arcee AI	87.21%
33	GPT-4o	OpenAI	87.2%
34	Mistral Large 3	Mistral	87.1%
35	LongCat-Flash-Omni		86.81%
36	Claude 3 Opus	Anthropic	86.8%
37	GPT-4 Turbo	OpenAI	86.7%
38	Grok 4	xAI	86.6%
39	MiniMax M2.5	MiniMax	86.5%
40	Qwen2.5-72B-Instruct	Alibaba	86.1%
41	Kimi K2.5	Moonshot AI	86%
42	o3-mini	OpenAI	85.9%
43	Gemini 1.5 Pro	Google	85.9%
44	Step-3.5-Flash Base		85.8%
45	o1-mini	OpenAI	85.2%
46	Qwen 3 14B	Alibaba	84.3%
47	Phi-4 14B	Microsoft	83.9%
48	GPT-4o mini	OpenAI	82%
49	Llama 3.1 70B	Meta	82%
50	Qwen3-Omni-30B-A3B-Base-202507		81.69%
51	Qwen3-VL-8B-Instruct	Qwen	80.7%
52	MiniCPM-o 4.5-Instruct		77%
53	Aria		73.3%
54	Apertus-70B-Instruct		69.6%
55	Llama 2 70B (5-shot)		68.9%
56	Chameleon 34B		65.8%
57	Apertus-70B		65.2%
58	LLaMA-65B		63.4%
59	OLMo-2-7B-1124 (olmOCR-peS2o)		61.1%
60	HRM-Text-1B		60.7%
61	BLT-Entropy 8B		57.4%
62	Helium		54.3%
63	BitNet b1.58 2B4T		53.17%
64	Moshi	Kyutai	49.7%

Source: hendrycks/test (MMLU) · Saturated benchmark. Small score deltas at the top (90–93%) are within noise; treat rankings as a cluster, not a strict order.

GPQA Diamond

198 expert-authored graduate-level questions in biology, chemistry, and physics. PhD-level specialists score ~65% on their own field. Designed to be impossible to Google.

#	Model	Vendor	accuracy
01	Gemini 3 Pro	Google	91.9%
02	Claude Opus 4.6	Anthropic	91.3%
03	Kimi K2.6		90.5%
04	Gemini 3 Flash	Google	90.4%
05	DeepSeek-V4-Pro Max	DeepSeek	90.1%
06	Claude Sonnet 4.6	Anthropic	89.9%
07	GPT-5	OpenAI	89%
08	Qwen3.5-397B-A17B	Alibaba	88.4%
09	DeepSeek-V4-Flash Max	DeepSeek	88.1%
10	Grok 4	xAI	88%
11	Qwen3.6-27B		87.8%
12	Kimi-K2.5	Moonshot.AI	87.6%
13	Qwen3.5-122B-A10B	Alibaba	86.6%
14	Gemini 2.5 Pro		86.4%
15	GLM-5.1		86.2%
16	GLM-5	Zhipu AI	86%
17	Qwen3.6-35B-A3B		86%
18	DeepSeek-V3.2-Speciale	DeepSeek	85.7%
19	GLM-4.7	Zhipu AI	85.7%
20	Qwen3.5-27B	Alibaba	85.5%
21	MiniMax-M2.5	MiniMaxAI	85.2%
22	Step-3.5-Flash PaCoRe		85%
23	Gemma 4 31B	Google	84.3%
24	Qwen3.5-35B-A3B	Alibaba	84.2%
25	Gemini 2.5 Pro	Google	84%
26	Qwen3.5-Omni-Plus		83.9%
27	Step-3.5-Flash		83.5%
28	Gemini 2.5 Flash	Google	82.8%
29	Gemini 2.5 Flash		82.8%
30	o3	OpenAI	82.8%
31	DeepSeek-V3.2	DeepSeek	82.4%
32	NVIDIA-Nemotron-3-Super-120B-A12B-BF16		79.23%
33	GLM-4.5	Zhipu AI	79.1%
34	o4-mini	OpenAI	77.6%
35	Qwen3-VL-235B-A22B-Thinking	Qwen	77.1%
36	Claude Opus 4	Anthropic	76.7%
37	o1	OpenAI	75.7%
38	GLM-4.5-Air	Zhipu AI	75%
39	o3-mini	OpenAI	74.9%
40	Claude Opus 4.5	Anthropic	74.9%
41	Qwen3-Coder-Next	Qwen	74.49%
42	Qwen3-VL-235B-A22B-Instruct	Qwen	74.3%
43	o1-preview	OpenAI	73.3%
44	Qwen3-Omni-Flash-Thinking		73.1%
45	NVIDIA-Nemotron-3-Nano-30B-A3B-BF16		73%
46	DeepSeek R1	DeepSeek	71.5%
47	Qwen3-235B-A22B	Alibaba	71.1%
48	Qwen3-235B-A22B	Alibaba	71.1%
49	ZAYA1-8B	Z.ai	71%
50	Claude Sonnet 4	Anthropic	70%
51	Llama 4 Maverick	Meta	69.8%
52	GPT-4.5 Preview	OpenAI	69.5%
53	MiMo-V2.5-Pro		66.7%
54	GPT-4.1 mini	OpenAI	66.4%
55	GPT-4.1	OpenAI	66.3%
56	Trinity Large Preview	Arcee AI	63.32%
57	o1-mini	OpenAI	60%
58	Claude 3.5 Sonnet	Anthropic	59.4%
59	Grok 2	xAI	56%
60	MiniMax-Text-01	MiniMax	54.4%
61	Llama 3 (405B, Instruct)	Meta	51.1%
62	Llama 3.1 405B	Meta	50.7%
63	Claude 3 Opus	Anthropic	50.4%
64	GPT-4o	OpenAI	49.9%
65	Qwen2.5-Plus		49.7%
66	GPT-4 Turbo	OpenAI	49.3%
67	Qwen2.5-72B-Instruct	Alibaba	49%
68	Qwen2.5-VL-72B		49%
69	Gemini 1.5 Pro	Google	46.2%
70	Gemma 3 (27B, IT)		42.4%
71	Llama 3.1 70B	Meta	41.7%
72	Step-3.5-Flash Base		41.7%
73	GPT-4o mini	OpenAI	40.2%
74	Qwen3-VL-8B-Instruct	Qwen	34.7%

Source: arXiv:2311.12022 · Human expert baseline (non-specialist): 34%. PhD specialist: ~65%.

§ 05 · Math / Reasoning

Math & reasoning.

Olympiad-style short answer, released after model training cutoffs.

AIME 2025

The 2025 American Invitational Mathematics Examination: 30 olympiad-style short-answer problems drawn after most 2024-era model training cutoffs. A primary frontier-math signal in recent reasoning-model reports.

#	Model	Vendor	accuracy
01	Step-3.5-Flash PaCoRe		99.9%
02	Step-3.5-Flash		97.3%
03	Kimi-K2.5	Moonshot.AI	96.1%
04	DeepSeek-V3.2-Speciale	DeepSeek	96%
05	SU-01		94.6%
06	Intern-S1-Pro	Shanghai AI Lab	93.1%
07	DeepSeek-V3.2	DeepSeek	93.1%
08	o4-mini	OpenAI	92.7%
09	Qwen3-VL-235B-A22B-Thinking	Qwen	89.7%
10	NVIDIA-Nemotron-3-Nano-30B-A3B-BF16		89.1%
11	Gemini 2.5 Pro		88%
12	o3	OpenAI	86.7%
13	Gemini 2.5 Pro	Google	86.7%
14	Qwen3-Coder-Next	Qwen	83.07%
15	Qwen3-235B-A22B	Alibaba	81.5%
16	Claude Opus 4.5	Anthropic	80%
17	Qwen3-VL-235B-A22B-Instruct	Qwen	74.7%
18	Qwen3-Omni-Flash-Thinking		74%
19	DeepSeek R1	DeepSeek	72%
20	Gemini 2.5 Flash		72%
21	Qwen3-VL-8B-Instruct	Qwen	45.9%
22	Trinity Large Preview	Arcee AI	24.36%

Source: maa.org/aime · Small test set (30 problems) — a single swing is ~3.3%. Numbers below are pass@1 unless otherwise noted.

§ 06 · Coding

Code.

Contest-style programming. Elo-rated or pass@1 on held-out problems.

LiveCodeBench Pro

The 2026 Elo-rated successor to classic LCB. Built by Olympiad medalists from continuously-updated Codeforces, ICPC and IOI problems. Each LLM is treated as a virtual Codeforces contestant and fit to a Bayesian MAP Elo on the standard Codeforces scale (~800 novice to ~3800 top human).

#	Model	Vendor	Elo
01	Gemini 3.1 Pro	Google	2887
02	Gemini 3 Pro	Google	2439
03	GPT-5	OpenAI	2176
04	o4-mini	OpenAI	2092
05	Gemini 2.5 Pro	Google	1769
06	Qwen3-235B-A22B	Alibaba	1673
07	Claude Sonnet 4.5	Anthropic	1412
08	Gemini 2.5 Flash	Google	1288
09	DeepSeek R1	DeepSeek	1161
10	o3	OpenAI	1010

Source: livecodebenchpro.com · Elo rating comparable to the Codeforces human scale. Top human contestants sit around 3800; the strongest model on the board is Gemini 3 Pro at 2439.

LiveCodeBench (classic)

Classic pass@1 LiveCodeBench — continuously updated with new contest problems from LeetCode, Codeforces, and AtCoder. Largely superseded by LCB Pro for frontier models, but preserved here for historical comparison across older models.

#	Model	Vendor	pass-1
01	DeepSeek-V4-Pro Max	DeepSeek	93.5%
02	Gemini 3 Pro Preview	Google	91.7%
03	DeepSeek-V4-Flash Max	DeepSeek	91.6%
04	Gemini 3 Flash	Google	90.8%
05	Kimi K2.6		89.6%
06	DeepSeek-V3.2-Speciale	DeepSeek	88.7%
07	Kimi-K2.5	Moonshot.AI	85%
08	GPT-5	OpenAI	85%
09	Qwen3.6-27B		83.9%
10	Qwen3.5-397B-A17B	Alibaba	83.6%
11	DeepSeek-V3.2	DeepSeek	83.3%
12	NVIDIA-Nemotron-3-Super-120B-A12B-BF16		81.19%
13	Qwen3.6-35B-A3B		80.4%
14	Gemma 4 31B	Google	80%
15	Grok 4	xAI	79%
16	Gemini 2.5 Pro	Google	75.6%
17	Intern-S1-Pro	Shanghai AI Lab	74.3%
18	Gemini 2.5 Pro		74.2%
19	DeepSeek-R1-0528	DeepSeek	73.3%
20	GLM-4.5	Zhipu AI	72.9%
21	o4-mini	OpenAI	72.8%
22	Qwen3-235B-A22B	Alibaba	70.7%
23	GLM-4.5-Air	Zhipu AI	70.7%
24	Qwen3-235B-A22B	Alibaba	70.7%
25	Qwen3-VL-235B-A22B-Thinking	Qwen	70.1%
26	NVIDIA-Nemotron-3-Nano-30B-A3B-BF16		68.3%
27	o3-mini	OpenAI	66.9%
28	DeepSeek R1	DeepSeek	65.9%
29	o3	OpenAI	65.3%
30	DeepSeek-R1-Distill-Llama-70B	DeepSeek	65.2%
31	Gemini 2.5 Flash	Google	63.9%
32	Kimi k1.5	Moonshot AI	62.5%
33	DeepSeek-R1-Distill-Qwen-32B	DeepSeek	62.1%
34	Gemini 2.5 Flash		59.3%
35	Qwen3-Coder-Next	Qwen	58.93%
36	Claude Opus 4	Anthropic	57.8%
37	Qwen2.5-72B-Instruct		55.5%
38	GPT-4.1	OpenAI	54.4%
39	Qwen3-VL-235B-A22B-Instruct	Qwen	54.3%
40	Claude Sonnet 4	Anthropic	52.8%
41	DeepSeek-v3-0324	DeepSeek	49.2%
42	DeepSeek-V3	DeepSeek	49.2%
43	GPT-4.1 mini	OpenAI	48.3%
44	Qwen2.5-Coder 32B	Alibaba	47.8%
45	Llama 4 Maverick	Meta	43.4%
46	DeepSeek-Coder-V2-Instruct	DeepSeek	43.4%
47	GPT-4o	OpenAI	40.8%
48	Qwen3-VL-8B-Instruct	Qwen	39.3%
49	Gemma-3-27b	Google	39%
50	Llama-4-Scout	Meta	32.8%
51	Gemma 3 12B IT	Google DeepMind	32%
52	Gemma 3 (27B, IT)		29.7%
53	Codestral 22B	Mistral	29.5%
54	Gemma 3 4B IT	Google DeepMind	23%

Source: livecodebench.github.io · Problems released after model training cutoffs to prevent contamination.

§ 07 · Agentic / Tools

Agentic & tools.

Multi-turn tasks using real tools and databases; pass = full resolution.

Tau2-Bench

Simulates real customer service interactions — agents use tools and databases to resolve tasks in retail and airline domains across multi-turn dialogues. Pass rate = task fully resolved.

#	Model	Vendor	accuracy
01	GLM-5	Zhipu AI	89.7%
02	Step-3.5-Flash		88.2%
03	Qwen3.5-397B-A17B	Alibaba	86.7%
04	Qwen3.5-35B-A3B	Alibaba	81.2%
05	Intern-S1-Pro	Shanghai AI Lab	80.9%
06	DeepSeek-V3.2	DeepSeek	80.3%
07	Qwen3.5-122B-A10B	Alibaba	79.5%
08	Qwen3.5-27B	Alibaba	79%
09	Claude Opus 4.5	Anthropic	79%
10	Ling-2.6-1T		78.36%
11	SenseNova-U1-A3B-MoT	SenseTime	75.39%
12	GPT-5.2	OpenAI	73%
13	Gemini 3 Pro	Google	69%
14	Claude Sonnet 4.5	Anthropic	63%
15	NVIDIA-Nemotron-3-Super-120B-A12B-BF16		61.15%
16	GPT-5.1	OpenAI	59%
17	Gemini 2.5 Pro	Google	54%
18	Claude 3.7 Sonnet	Anthropic	47%
19	GPT-4o	OpenAI	36%

Source: sierra-research/tau2-bench · Average across 3 seeds per model.

§ 08 · Frontier

Frontier difficulty.

Designed to remain unsaturated for years. Even the leaders score low.

Humanity's Last Exam (HLE)

3,000 extremely hard questions across math, science, law, and humanities — contributed by domain experts worldwide. Designed to remain unsaturated for years. No tools allowed in this variant.

#	Model	Vendor	accuracy
01	Kimi K2.6		54%
02	MiMo-V2.5-Pro		48%
03	Gemini 3.1 Pro	Google	46.44%
04	GPT-5.4 Pro	OpenAI	44.32%
05	Muse Spark	Meta	40.56%
06	Gemini 3 Pro	Google	38.3%
07	DeepSeek-V4-Pro Max	DeepSeek	37.7%
08	Gemini 3 Pro Preview	Google	37.52%
09	GPT-5.4	OpenAI	36.24%
10	Claude Opus 4.7	Anthropic	36.2%
11	DeepSeek-V4-Flash Max	DeepSeek	34.8%
12	Claude Opus 4.6	Anthropic	34.44%
13	GPT-5 Pro	OpenAI	31.64%
14	GLM-5.1		31%
15	DeepSeek-V3.2-Speciale	DeepSeek	30.6%
16	GLM-5	Zhipu AI	30.5%
17	Kimi-K2.5	Moonshot.AI	30.1%
18	Qwen3.5-397B-A17B	Alibaba	28.7%
19	Step-3.5-Flash PaCoRe		27.9%
20	GPT-5.2	OpenAI	27.8%
21	Gemma 4 31B	Google	26.5%
22	GPT-5	OpenAI	25.32%
23	GPT-5	OpenAI	25.3%
24	Claude Opus 4.5	Anthropic	25.2%
25	DeepSeek-V3.2	DeepSeek	25.1%
26	GLM-4.7	Zhipu AI	24.8%
27	Grok 4	xAI	24.5%
28	Kimi K2.5	Moonshot AI	24.37%
29	Qwen3.6-27B		24%
30	GPT-5.1	OpenAI	23.68%
31	Step-3.5-Flash		23.1%
32	Gemini 2.5 Pro	Google	21.64%
33	Gemini 2.5 Pro		21.6%
34	Gemini 2.5 Pro	Google	21.6%
35	Qwen3.6-35B-A3B		21.4%
36	o3	OpenAI	20.32%
37	GPT-5 mini	OpenAI	19.44%
38	GPT-5 mini	OpenAI	19.4%
39	MiniMax-M2.5	MiniMaxAI	19.4%
40	Claude Opus 4.6	Anthropic	19%
41	NVIDIA-Nemotron-3-Super-120B-A12B-BF16		18.26%
42	o4-mini	OpenAI	18.08%
43	GLM-4.5	Zhipu AI	14.4%
44	Claude Sonnet 4.5	Anthropic	13.72%
45	Claude 4.5 Sonnet	Anthropic	13.7%
46	Claude Sonnet 4.6	Anthropic	13.2%
47	Gemini 2.5 Flash	Google	12.1%
48	Gemini 2.5 Flash	Google	12.08%
49	Claude Opus 4.1	Anthropic	11.52%
50	Gemini 2.5 Flash		11%
51	Claude Opus 4	Anthropic	10.72%
52	GLM-4.5-Air	Zhipu AI	10.6%
53	NVIDIA-Nemotron-3-Nano-30B-A3B-BF16		10.6%
54	Gemini 3.1 Flash-Lite	Google	8.64%
55	DeepSeek R1	DeepSeek	8.5%
56	GLM-4.5	Zhipu AI	8.32%
57	GLM-4.5-Air	Zhipu AI	8.12%
58	o1 Pro	OpenAI	8.12%
59	Claude 3.7 Sonnet	Anthropic	8.04%
60	o1	OpenAI	8%
61	o1	OpenAI	7.96%
62	Claude Sonnet 4	Anthropic	7.76%
63	Gemini 2.0 Flash Thinking	Google	6.56%
64	Llama 4 Maverick	Meta	5.68%
65	GPT-4.5 Preview	OpenAI	5.44%
66	GPT-4.1	OpenAI	5.4%
67	Gemini 1.5 Pro	Google	4.6%
68	GPT-4.1 mini	OpenAI	4.6%
69	Mistral-Medium-3	Mistral	4.52%
70	Nova Pro	Amazon	4.4%
71	Claude 3.5 Sonnet	Anthropic	4.08%
72	Nova Lite	Amazon	3.64%
73	GPT-4o	OpenAI	2.72%
74	GPT-4o	OpenAI	2.7%

Source: agi.safe.ai · Leaderboard updated from live source.

§ 09 · Browse

By capability.

A shortcut into deeper leaderboards and per-task pages. All links resolve to live registry pages.

Multi-step, frontier-difficulty, GPQA and HLE.

Reasoning →

AIME 2025, olympiad-style short answer.

Math →

LiveCodeBench, pass@1 on held-out contest problems.

Code generation →

MMLU and MMLU-Pro — breadth across 57 subjects.

Knowledge →

SWE-bench, Tau2-Bench, tool-use under real constraints.

Agentic →

The full index of text-in, text-out tasks.

All LLM datasets →

§ 10 · Deep dives

By benchmark family.

Editorial pages with current rankings, eval methodology, and what the score actually means.

Benchmark ladder: 1M–7B →

Which evals work at each model scale — from bits-per-byte and BLiMP to MMLU, GSM8K, and HumanEval.

Small English language models →

A same-harness ranking plus a unified rerun roster for compact base models around the 125M class.

LiveCodeBench, SWE-bench Verified, HumanEval+, MBPP — pass@1 across the coding leaderboards.

Coding benchmarks →

The two saturating Python micro-benchmarks — what they still tell you and what they don’t.

HumanEval & MBPP →

AIME, MATH, Omni-MATH — frontier models on olympiad-style problems.

Math benchmarks →

Grade-school math word problems — the canonical reasoning benchmark.

GSM8K →

GPQA, HLE, ARC-AGI — what frontier-difficulty actually means.

Reasoning benchmarks →

Llama, Qwen, DeepSeek, Mistral — the open frontier vs the closed.

Open-weight models →

§ 11 · Related

Keep reading.

Adjacent sections of the registry.

SWE-bench, Terminal-Bench, tool-use and the trust problem in agent evals.

Agentic →

The pass@1 era and what comes after.

Code generation →

Long-form guide comparing code models in production.

Guide · Code models →