Model card
Claude 3 Opus.
Anthropicapi
Most capable Claude 3 model, March 2024. Supports image input. Source: Anthropic Claude 3 family announcement.
§ 02 · Benchmarks
Every benchmark Claude 3 Opus has a recorded score for.
| # | Benchmark | Area · Task | Metric | Value | Rank | Date | Source |
|---|---|---|---|---|---|---|---|
| 01 | BIG-Bench Hard | Reasoning · Multi-step Reasoning | accuracy | 86.8% | #6 | — | source ↗ |
| 02 | MMMU | Multimodal · Visual Question Answering | accuracy | 59.4% | #23 | 2024-03-04 | source ↗ |
| 03 | GSM8K | Reasoning · Mathematical Reasoning | accuracy | 95.0% | #24 | 2024-03-01 | source ↗ |
| 04 | HumanEval | Computer Code · Code Generation | pass@1 | 84.9% | #31 | — | source ↗ |
| 05 | PLCC | Natural Language Processing · Polish Cultural Competency | art-and-entertainment | 73.0% | #31 | — | source ↗ |
| 06 | MMLU | Reasoning · Commonsense Reasoning | accuracy | 86.8% | #36 | — | source ↗ |
| 07 | PLCC | Natural Language Processing · Polish Cultural Competency | history | 86.0% | #36 | — | source ↗ |
| 08 | MATH | Reasoning · Mathematical Reasoning | accuracy | 60.1% | #40 | — | source ↗ |
| 09 | PLCC | Natural Language Processing · Polish Cultural Competency | culture-and-tradition | 76.0% | #46 | — | source ↗ |
| 10 | PLCC | Natural Language Processing · Polish Cultural Competency | average | 73.8% | #49 | — | source ↗ |
| 11 | PLCC | Natural Language Processing · Polish Cultural Competency | geography | 80.0% | #58 | — | source ↗ |
| 12 | PLCC | Natural Language Processing · Polish Cultural Competency | vocabulary | 62.0% | #60 | — | source ↗ |
| 13 | PLCC | Natural Language Processing · Polish Cultural Competency | grammar | 66.0% | #61 | — | source ↗ |
| 14 | GPQA Diamond | Reasoning · Multi-step Reasoning | accuracy | 50.4% | #63 | — | source ↗ |
Rank column shows this model’s position vs all other models scored on the same benchmark + metric (competitors after the slash). #1 in red means current SOTA. Sorted by rank, then newest result.
§ 03 · Strengths by area
Where Claude 3 Opus actually performs.
§ 04 · Papers
1 paper with results for Claude 3 Opus.
- 2024-03-04· Multimodal· 1 result
Claude 3 Model Family (Haiku, Sonnet, Opus)
§ 05 · Related models
Other Anthropic models scored on Codesota.
Claude Opus 4
Undisclosed params · 14 results · 2 SOTA
Claude Sonnet 5
Undisclosed params · 2 results · 2 SOTA
Claude Sonnet 4
11 results · 1 SOTA
Claude Opus 4.5
4 results · 1 SOTA
Claude Opus 4.7
2 results · 1 SOTA
Claude Mythos Preview
1 result · 1 SOTA
Claude 3.5 Sonnet
Undisclosed params · 28 results
Claude Opus 4.5
Undisclosed params · 12 results
§ 06 · Sources & freshness
Where these numbers come from.
sdadas/PLCC
7
results
openai-simple-evals
4
results
llm-stats-bbh
1
result
arxiv
1
result
gsm8k-shadow-page
1
result
9 of 14 rows marked verified. · first result 2024-03-01, latest 2024-03-04.