Logical Reasoning
Solving logic puzzles and deductive problems.
Logical Reasoning is a key task in reasoning. Below you will find the standard benchmarks used to evaluate models, along with current state-of-the-art results.
Benchmarks & SOTA
LogiQA
LogiQA
8,678 logical reasoning questions from National Civil Servants Examinations of China.
State of the Art
GPT-4o
OpenAI
56.3
accuracy
ReClor
Reading Comprehension Dataset Requiring Logical Reasoning
6,138 reading comprehension questions requiring logical reasoning from GMAT/LSAT exams.
State of the Art
GPT-4o
OpenAI
72.4
accuracy
Related Tasks
Mathematical Reasoning
Solving math word problems (GSM8K, MATH, Minerva).
Commonsense Reasoning
Reasoning about everyday situations (CommonsenseQA, HellaSwag).
Multi-step Reasoning
Complex reasoning requiring multiple inference steps (HotpotQA).
Arithmetic Reasoning
Performing arithmetic calculations and solving equations.