Home/Browse/Computer Code/Code Generation/SWE-Bench Verified

SWE-Bench Verified

Name: SWE-Bench Verified Benchmark Results
Creator: Unknown
License: https://creativecommons.org/licenses/by/4.0/

Unknown

500 manually verified GitHub issues confirmed solvable by human engineers. High-quality subset of SWE-bench.

Paper Leaderboard

Benchmark Stats

Models29

Papers29

Metrics1

SOTA History

Not enough data to show trend.

resolve-rate

Higher is better

Rank	Model	Source	Score	Year	Paper
1	Claude Opus 4.5 First model to break 80%. With agent scaffolding.	Editorial	80.9	2026	Source
2	Claude Opus 4.6 Simple bash + file edit scaffold.	Editorial	80.8	2026	Source
3	Gemini 3.1 Pro With custom agent scaffold.	Editorial	80.6	2026	Source
4	GPT-5.2 Thinking GPT-5.2 with thinking mode and agent scaffold.	Editorial	80	2026	Source
5	Claude Sonnet 4.6 Simple bash + file edit scaffold.	Editorial	79.6	2026	Source
6	Gemini 3 Flash With custom agent scaffold.	Editorial	78	2026	Source
7	claude-sonnet-45	Editorial	77.2	2026	Source
8	GPT-5.1 GPT-5.1 with agent scaffold.	Editorial	76.3	2026	Source
9	Gemini 3 Pro With custom agent scaffold.	Editorial	76.2	2026	Source
10	GPT-5 GPT-5 with agent scaffold.	Editorial	74.9	2026	Source
11	Claude Haiku 4.5 128K thinking budget. Simple scaffold.	Editorial	73.3	2026	Source
12	claude-sonnet-4	Editorial	72.2	2026	Source
13	Devstral 2 Devstral 2 with SWE-agent.	Editorial	72.2	2026	Source
14	o3	Editorial	69.1	2026	Source
15	o4-mini With agent scaffolding.	Editorial	68.1	2026	Source
16	Grok 3 With custom agent scaffold.	Editorial	63.8	2026	Source
17	gemini-25-pro	Editorial	63.8	2026	Source
18	claude-37-sonnet	Editorial	62.3	2026	Source
19	DeepSeek R1-0528 Self-reported score with Agentless framework.	Editorial	57.6	2026	Source
20	o3-mini	Editorial	55.8	2026	Source
21	gpt-41	Editorial	54.6	2026	Source
22	Gemini 2.5 Flash With custom agent scaffold.	Editorial	54	2026	Source
23	deepseek-r1	Editorial	49.2	2026	Source
24	claude-35-sonnet	Editorial	49	2026	Source
25	o1	Editorial	48.9	2026	Source
26	deepseek-v3	Editorial	42	2026	Source
27	gpt-4o	Editorial	41.2	2026	Source
28	Claude 3.5 Haiku Standard scaffold.	Editorial	40.6	2026	Source
29	deepseek-v25	Editorial	37	2026	Source

Submit a Result

Back to Code Generation