Model card
o3-mini.
OpenAIapi
§ 02 · Benchmarks
Every benchmark o3-mini has a recorded score for.
| # | Benchmark | Area · Task | Metric | Value | Rank | Date | Source |
|---|---|---|---|---|---|---|---|
| 01 | HumanEval | Computer Code · Code Generation | pass@1 | 96.3% | #2 | — | source ↗ |
| 02 | MBPP | Computer Code · Code Generation | pass@1 | 93.3% | #2 | — | source ↗ |
| 03 | MATH | Reasoning · Mathematical Reasoning | accuracy | 97.9% | #3 | — | source ↗ |
| 04 | LiveCodeBench | Computer Code · Code Generation | pass@1 | 66.9% | #9 | 2024-03-12 | source ↗ |
| 05 | SWE-Bench Verified | Computer Code · Code Generation | resolve-rate | 55.8% | #30 | — | source ↗ |
| 06 | GPQA Diamond | Reasoning · Multi-step Reasoning | accuracy | 74.9% | #39 | — | source ↗ |
| 07 | MMLU | Reasoning · Commonsense Reasoning | accuracy | 85.9% | #42 | — | source ↗ |
| 08 | SWE-bench Verified | Agentic AI · SWE-bench | resolve-rate | 49.3% | #66 | — | source ↗ |
Rank column shows this model’s position vs all other models scored on the same benchmark + metric (competitors after the slash). #1 in red means current SOTA. Sorted by rank, then newest result.
§ 03 · Strengths by area
Where o3-mini actually performs.
§ 04 · Papers
1 paper with results for o3-mini.
- 2024-03-12· Computer Code· 1 result
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
§ 05 · Related models
Other OpenAI models scored on Codesota.
§ 06 · Sources & freshness
Where these numbers come from.
openai-simple-evals
4
results
official-model-card
1
result
official-leaderboard
1
result
swebench-leaderboard
1
result
editorial
1
result
3 of 8 rows marked verified.