Codesota · Reasoning · Commonsense Reasoning · HellaSwagTasks/Reasoning/Commonsense Reasoning
Commonsense Reasoning · benchmark dataset · 2019 · EN

HellaSwag.

70K sentence completion problems testing commonsense natural language inference.

Paper Download datasetSubmit a result
§ 01 · Leaderboard

Best published scores.

17 results indexed across 1 metric. Shaded row marks current SOTA; ties broken by submission date.


Primary
accuracy · higher is better
accuracy· primary
17 rows
#ModelOrgSubmittedPaper / codeaccuracy
01GPT-4oAPIOpenAIDec 2025openai-blog95.30
02Gemini 1.5 ProAPIGoogleDec 2025google-blog92.50
03Step-3.5-Flash BaseFeb 2026Step 3.5 Flash: Open Frontier-Level Intelligence with 11… · code90.20
04Trinity Large Base (5-shot)Feb 2026Arcee Trinity Large Technical Report · code90.11
05Claude 3.5 SonnetAPIAnthropicDec 2025anthropic-blog89
06Llama 3.1 405BOpenMetaMar 2026meta-modelcard89
07Llama 3 70BOpenMetaDec 2025meta-blog88
08LLaMA-65BFeb 2023LLaMA: Open and Efficient Foundation Language Models · code84.20
09Chameleon 34BMay 2024Chameleon: Mixed-Modal Early-Fusion Foundation Models · code82.70
10BLT-Entropy 8BDec 2024Byte Latent Transformer: Patches Scale Better Than Token… · code80.60
11Apertus-70B-InstructSep 2025Apertus: Democratizing Open and Compliant LLMs for Globa… · code78.10
12HeliumSep 2024Moshi: a speech-text foundation model for real-time dial… · code76.30
13SmoLM2 (1.7B)Feb 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of … · code68.70
14BitNet b1.58 2B4TApr 2025BitNet b1.58 2B4T Technical Report · code68.44
15Apertus-70BSep 2025Apertus: Democratizing Open and Compliant LLMs for Globa… · code64
16HRM-Text-1BMay 2026pwc-dump · code63.40
17OLMo-2-7B-1124 (olmOCR-peS2o)Feb 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Visio… · code62.60
Fig 2 · Rows sorted by score within each metric. Shaded row marks SOTA. Dates reflect model or paper release where available, otherwise the date Codesota accessed the source.
§ 03 · Progress

2 steps
of state of the art.

Each row below marks a model that broke the previous record on accuracy. Intermediate submissions are kept in the leaderboard above; only SOTA-setting entries are re-listed here.

Higher scores win. Each subsequent entry improved upon the previous best.

SOTA line · accuracy
  1. Feb 27, 2023LLaMA-65B84.20
  2. Dec 17, 2025GPT-4oOpenAI95.30
Fig 3 · SOTA-setting models only. 2 entries span Feb 2023 Dec 2025.
§ 04 · Literature

10 papers
tied to this benchmark.

Every paper below corresponds to at least one row in the leaderboard above. Click through for the arXiv preprint and, when available, the reference implementation.

§ 06 · Contribute

Have a score that beats
this table?

Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.

Submit a result Read submission guide
What a submission needs
  • 01A public checkpoint or API endpoint
  • 02A reproduction script with frozen commit + seed
  • 03Declared evaluation environment (Python, deps)
  • 04One row per metric declared by this dataset
  • 05A contact so we can follow up on discrepancies