CodeSOTA · Experiment · E-0009Portable protocol · public evidence graph
GSM8K matched-model audit
Measure the visible score range for the same three frontier models on GSM8K.
01 · Design
Method before metrics.
Resolve the three verified chain-of-thought benchmark_result rows live and compute max minus min.
- Hypothesis
- H-0013 · GSM8K produces at least a five-point range across o3, o4-mini and Gemini 2.5 Pro in the verified snapshot.
02 · Protocol
The comparison contract.
A reproduction matches this protocol. A fork changes it and declares the deviation.
- dataset
- gsm8k
- controlled Variables
- model set · metric · chain-of-thought family
- independent Variable
- model
- benchmarks
- accuracy
- screening threshold points
- 5
03 · Evidence outputs
What this experiment produced or evaluated.
- Uses
- benchmark:gsm8k
Contribute
Change the evidence, not just the discussion.
Start locally, inspect the portable manifest, and sign in only if you choose to publish.
Add evidence
Attach a run to this protocol.
Record the measurement and its provenance. New evidence is unverified and pending review until CodeSOTA checks it.
04 · Record
Stable, portable, attributable.
- Stable ID
- E-0009
- Visibility
- public
- Created
- 2026-08-20
- Updated
- 2026-08-20