Model card
o1.
OpenAIapi
§ 02 · Benchmarks
Every benchmark o1 has a recorded score for.
| # | Benchmark | Area · Task | Metric | Value | Rank | Date | Source |
|---|---|---|---|---|---|---|---|
| 01 | MMLU | Reasoning · Commonsense Reasoning | accuracy | 91.8% | #3 | 2024-12-05 | source ↗ |
| 02 | RE-Bench | Agentic AI · RE-Bench | normalized-score | 0.2% | #3 | 2024-11-22 | source ↗ |
| 03 | HCAST | Agentic AI · HCAST | success-rate | 28.0% | #4 | 2025-04-01 | source ↗ |
| 04 | METR Time Horizon | Agentic AI · Time Horizon | task-horizon-minutes | 4.0% | #4 | 2025-04-01 | source ↗ |
| 05 | MATH | Reasoning · Mathematical Reasoning | accuracy | 96.4% | #8 | — | source ↗ |
| 06 | GSM8K | Reasoning · Mathematical Reasoning | accuracy | 97.8% | #13 | 2024-09-01 | source ↗ |
| 07 | GSM8K | Reasoning · Mathematical Reasoning | accuracy | 97.8% | #13 | 2024-09-01 | source ↗ |
| 08 | SWE-Bench Verified | Computer Code · Code Generation | resolve-rate | 48.9% | #34 | — | source ↗ |
| 09 | GPQA Diamond | Reasoning · Multi-step Reasoning | accuracy | 75.7% | #37 | — | source ↗ |
| 10 | HLE | Reasoning · Multi-step Reasoning | accuracy | 8.0% | #60 | — | |
| 11 | HLE | Reasoning · Multi-step Reasoning | accuracy | 8.0% | #61 | — | source ↗ |
| 12 | SWE-bench Verified | Agentic AI · SWE-bench | resolve-rate | 41.0% | #72 | — | source ↗ |
Rank column shows this model’s position vs all other models scored on the same benchmark + metric (competitors after the slash). #1 in red means current SOTA. Sorted by rank, then newest result.
§ 03 · Strengths by area
Where o1 actually performs.
§ 04 · Papers
2 papers with results for o1.
- 2025-04-01· Agentic AI· 2 results
METR: Measuring Autonomy in AI Systems (2025 Update)
- 2024-11-22· Agentic AI· 1 result
RE-Bench: Evaluating Frontier AI R&D Capabilities of Language Model Agents Against Human Experts
§ 05 · Related models
Other OpenAI models scored on Codesota.
§ 06 · Sources & freshness
Where these numbers come from.
openai-simple-evals
3
results
official-leaderboard
2
results
editorial
2
results
arxiv
1
result
gsm8k-shadow-page
1
result
gsm8k-shadow-page-timeline
1
result
swebench-leaderboard
1
result
scale-hle-official
1
result
6 of 12 rows marked verified. · first result 2024-09-01, latest 2025-04-01.