CodeSOTA · Clickable claim · C-0011Evidence snapshot 2026-08-20
Public evidence statement

ARC-AGI-1 spans 31.4 percentage points across the matched o3, o4-mini and Gemini 2.5 Pro CodeSOTA snapshot.

Claim
C-0011
Verdict
supported
Verification
CodeSOTA verified
Evidence
3 attached records
Authors
Kacper Wikiel

What this claim answers.

Hypothesis
H-0011 · ARC-AGI-1 separates the matched frontier snapshot

Inspect the rows beneath the sentence.

Current registry values are resolved live when available and compared with the captured claim snapshot. Provenance remains visible when resolution falls back to the snapshot.

benchmark-result:3520

o3 on ARC-AGI-1

CodeSOTA verified
Model
o3
Benchmark
ARC-AGI-1
Metric
accuracy
Value
87.5

Verified o3 ARC-AGI-1 row.

Registry note: Public evaluation set (400 tasks). High-compute setting (900 samples/task). Source: ARC Prize leaderboard (Dec 2024).

CodeSOTA benchmark_resultsobserved 2026-03-27unchanged since snapshotOpen registry evidence →Primary source ↗
benchmark-result:3522

o4-mini on ARC-AGI-1

CodeSOTA verified
Model
o4-mini
Benchmark
ARC-AGI-1
Metric
accuracy
Value
79.0

Verified o4-mini ARC-AGI-1 row.

Registry note: Public evaluation set. Source: ARC Prize leaderboard (April 2025).

CodeSOTA benchmark_resultsobserved 2026-03-27unchanged since snapshotOpen registry evidence →Primary source ↗
benchmark-result:3523

Gemini 2.5 Pro on ARC-AGI-1

CodeSOTA verified
Model
Gemini 2.5 Pro
Benchmark
ARC-AGI-1
Metric
accuracy
Value
56.1

Verified Gemini 2.5 Pro ARC-AGI-1 row.

Registry note: Public evaluation set. Source: Gemini 2.5 Pro technical report (April 2025).

CodeSOTA benchmark_resultsobserved 2026-03-27unchanged since snapshotOpen registry evidence →Primary source ↗

Verification is not the same as reproduction.

codesota verified

“CodeSOTA verified” means the evidence row and its source have passed the registry’s verification path. It does not, by itself, mean an independent team reran the protocol. Reproduction is recorded separately in the claim verdict and experiment graph.

A stable public object.

Created by
Kacper Wikiel
Created
2026-08-20
Snapshot
2026-08-20