CodeSOTA · Experiment · E-0003Portable protocol · public evidence graph
Why this run exists

Mistral OCR 3 on OmniDocBench

Independently measure end-to-end document parsing quality and preserve component-level evidence.

Fork this experiment →Open RQ-0002
Experiment
E-0003
Status
completed
Outcome
supported
Forks
5 public or local starts recorded
Owner
CodeSOTA evaluation programme
01 · Design

Method before metrics.

Run the full 1,355-image evaluation and retain composite, text, table, formula and reading-order results as separate CodeSOTA evidence rows.

Hypothesis
H-0004 · A single document parsing composite can hide practically decisive table or formula failures.
Hypothesis
H-0005 · Inspecting text, table, formula and reading-order metrics changes the preferred OCR system for task-specific deployments.
02 · Protocol

The comparison contract.

A reproduction matches this protocol. A fork changes it and declares the deviation.

model Family
Mistral OCR
dataset
OmniDocBench
benchmarks
composite · text edit distance · table TEDS · formula edit distance · reading order
controlled Variables
dataset version · full evaluation set
03 · Evidence outputs

What this experiment produced or evaluated.

Evaluates
model:mistral-ocr-3
Uses
benchmark:omnidocbench
Contribute

Change the evidence, not just the discussion.

Start locally, inspect the portable manifest, and sign in only if you choose to publish.

Add evidence

Attach a run to this protocol.

Record the measurement and its provenance. New evidence is unverified and pending review until CodeSOTA checks it.

04 · Record

Stable, portable, attributable.

Stable ID
E-0003
Visibility
public
Created
2026-04-23
Updated
2026-08-20