Model arena, by the numbers.

A prompt, two anonymous answers.

A user submits a prompt. Two unnamed models answer side by side. The user picks the better response without knowing which model is which. Bias from brand, cost, or training-set advertising drops out.

Votes aggregate via Bradley-Terry.

Each comparison updates both models' scores against their expected outcome — the same family of estimator as chess Elo. Millions of votes collapse into a single ordering per category, per model, per week.

Confidence intervals decide ties.

Each score carries a 95% interval. Models with overlapping intervals are statistically tied — not ranked. More votes tighten the interval, and the leaderboard only separates two models when the evidence is there.

What a preference ranking does — and doesn't — measure.

Arena scores tell you which model a typical user picks when shown two unnamed answers side by side. That is useful and it is real. It is not the same as accuracy on a graded task.

A model can win the arena while failing graded benchmarks, because users reward confident prose, formatting, and length. A model can lose the arena while acing graded benchmarks, because the user prompts skew toward creative tasks that graded suites don't cover. Treat the arena as one dimension of evidence, pair it with the task registry at /tasks, and route between them on your own workload.

We mirror arena.ai here, categorised and cross-linked. For the editorial methodology behind Codesota's wider registry — what counts as reproduced, what a retraction looks like, how we record dates — see /methodology.