Codesota · Benchmark · ARC-AGI-1Home/Leaderboards/Language & Knowledge/Logical Reasoning/ARC-AGI-1
Unknown

ARC-AGI-1.

400 evaluation tasks testing abstract visual reasoning. Created by François Chollet. Scores near human average (~85%) remained out of reach for LLMs until 2024.

Paper ↗Leaderboard ↓
§ 01 · Leaderboard

Results by metric.

Found a wrong score or missing run?
Use row edits to send a sourced correction into moderation.
Add / edit result ↗Report issue ↗

Accuracy

Accuracy is the reported evaluation metric for ARC-AGI-1. Codesota tracks published model scores on this metric so readers can compare state-of-the-art results across sources and model families.

Higher is better

Trust tiers for Accuracyverifiedpapervendorcommunityunverified
RankModelTrustScoreYearLinksFix
01o3
Public evaluation set (400 tasks). High-compute setting (900 samples/task). Source: ARC Prize leaderboard (Dec 2024).
verified87.52026Source ↗Looks wrong?
02o3 (high)
Public evaluation set (400 tasks). High-compute (1,024 candidates). Matches human average. Source: ARC Prize leaderboard (Dec 2024).
verified87.52026Source ↗Looks wrong?
03o4-mini
Public evaluation set. Source: ARC Prize leaderboard (April 2025).
verified792026Source ↗Looks wrong?
04Gemini 2.5 Pro
Public evaluation set. Source: Gemini 2.5 Pro technical report (April 2025).
verified56.12026Source ↗Looks wrong?
05Claude 3.7 Sonnet
Public evaluation set. Source: ARC Prize leaderboard (Feb 2025).
verified302026Source ↗Looks wrong?
§ 04 · Submit a result

Add to the leaderboard.

← Back to Logical Reasoning