The MMLU-Pro dataset contains 12K complex questions across various disciplines, including biology, business, chemistry, computer science, economics, engineering, math, physics, and psychology. It has 10 options per question, compared to the original MMLU's 4, making it more challenging. It also integrates more reasoning-focused problems, where Chain-of-Thought (CoT) results can be significantly higher than Perplexity (PPL).
Accuracy is the reported evaluation metric for MMLU-Pro. Codesota tracks published model scores on this metric so readers can compare state-of-the-art results across sources and model families.
Higher is better
Muted rows were not state of the art when published — an earlier or same-year result already scored better.
| Rank | Model | Trust | Score | Year | Links | Fix |
|---|---|---|---|---|---|---|
| 01 | MiniMax M2.1 | vendor | 88 | N/A | Code ↗Source ↗ | Looks wrong? |
| 02 | Intern S2 Preview | vendor | 88 | N/A | Code ↗ | Looks wrong? |
| 03 | Qwen3.5 397B A17B | vendor | 87.8 | N/A | Code ↗ | Looks wrong? |
| 04 | DeepSeek V4 Pro | vendor | 87.5 | N/A | Code ↗ | Looks wrong? |
| 05 | Kimi K2.5 | vendor | 87.1 | N/A | Code ↗ | Looks wrong? |
| 06 | NVIDIA Nemotron 3 Ultra 550B A55B BF16 | vendor | 86.8 | N/A | Code ↗ | Looks wrong? |
| 07 | NVIDIA Nemotron 3 Ultra 550B A55B NVFP4 | vendor | 86.8 | N/A | Code ↗ | Looks wrong? |
| 08 | Qwen3.5 122B A10B | vendor | 86.7 | N/A | Code ↗Source ↗ | Looks wrong? |
| 09 | DeepSeek V4 Flash | vendor | 86.4 | N/A | Code ↗ | Looks wrong? |
| 10 | Qwen3.6 27B | vendor | 86.2 | N/A | Code ↗ | Looks wrong? |
| 11 | Solar Open2 250B | vendor | 86.2 | N/A | Code ↗ | Looks wrong? |
| 12 | Qwen3.5 27B | vendor | 86.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 13 | GLM 5 | vendor | 86 | N/A | Code ↗Source ↗ | Looks wrong? |
| 14 | Qwen3.6 35B A3B | vendor | 85.2 | N/A | Code ↗ | Looks wrong? |
| 15 | DeepSeek R1 0528 | vendor | 85 | N/A | Code ↗ | Looks wrong? |
| 16 | GLM 4.5 | vendor | 84.6 | N/A | Code ↗Source ↗ | Looks wrong? |
| 17 | Step 3.5 Flash | vendor | 84.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 18 | DeepSeek R1 | vendor | 84 | N/A | Code ↗ | Looks wrong? |
| 19 | K EXAONE 236B A23B | vendor | 83.8 | N/A | Code ↗ | Looks wrong? |
| 20 | NVIDIA Nemotron 3 Super 120B A12B BF16 | vendor | 83.73 | N/A | Code ↗ | Looks wrong? |
| 21 | NVIDIA Nemotron 3 Super 120B A12B UD Q3 K M dequant bf16 vllm | vendor | 83.73 | N/A | Code ↗Source ↗ | Looks wrong? |
| 22 | NVIDIA Nemotron 3 Super 120B A12B UD Q4 K M dequant bf16 vllm | vendor | 83.73 | N/A | Code ↗Source ↗ | Looks wrong? |
| 23 | NVIDIA Nemotron 3 Super 120B A12B MXFP4 MOE dequant bf16 vllm | vendor | 83.73 | N/A | Code ↗Source ↗ | Looks wrong? |
| 24 | Intern S1 | vendor | 83.5 | N/A | Code ↗Source ↗ | Looks wrong? |
| 25 | EXAONE 4.5 33B | vendor | 83.3 | N/A | Code ↗ | Looks wrong? |
| 26 | Qwen3 235B A22B Instruct 2507 | vendor | 83 | N/A | Code ↗Source ↗ | Looks wrong? |
| 27 | Seed OSS 36B Instruct | vendor | 82.7 | N/A | Code ↗Source ↗ | Looks wrong? |
| 28 | LongCat Flash Chat | vendor | 82.7 | N/A | Code ↗Source ↗ | Looks wrong? |
| 29 | MiniMax M2 | vendor | 82 | N/A | Code ↗ | Looks wrong? |
| 30 | A.X K1 | vendor | 81.5 | N/A | Code ↗ | Looks wrong? |
| 31 | GLM 4.5 Air | vendor | 81.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 32 | DeepSeek V3 0324 | vendor | 81.2 | N/A | Code ↗ | Looks wrong? |
| 33 | MiniMax M1 40k | vendor | 81.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 34 | JoyAI LLM Flash | vendor | 81.02 | N/A | Code ↗ | Looks wrong? |
| 35 | Kimi K2 Instruct | vendor | 81 | N/A | Code ↗Source ↗ | Looks wrong? |
| 36 | Qwen3 30B A3B Thinking 2507 | vendor | 80.9 | N/A | Code ↗Source ↗ | Looks wrong? |
| 37 | gpt oss 120b | vendor | 80.8 | N/A | Code ↗Source ↗ | Looks wrong? |
| 38 | MiniMax M2.5 | vendor | 80.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 39 | ERNIE 4.5 300B A47B PT | vendor | 78.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 40 | LongCat Flash Lite | vendor | 78.29 | N/A | Code ↗ | Looks wrong? |
| 41 | MiniMax Text 01 | vendor | 75.7 | N/A | Code ↗Source ↗ | Looks wrong? |
| 42 | gpt oss 20b | vendor | 73.6 | N/A | Code ↗Source ↗ | Looks wrong? |
| 43 | GPT-4o | paper | 72.6 | 2024 | Paper ↗ | Looks wrong? |
| 44 | Qwen2.5 72B | vendor | 71.59 | N/A | Code ↗Source ↗ | Looks wrong? |
| 45 | phi 4 | vendor | 70.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 46 | Qwen3 4B Instruct 2507 | vendor | 69.6 | N/A | Code ↗ | Looks wrong? |
| 47 | ERNIE 4.5 300B A47B Base PT | vendor | 69.5 | N/A | Code ↗Source ↗ | Looks wrong? |
| 48 | Qwen2.5 32B | vendor | 69.23 | N/A | Code ↗Source ↗ | Looks wrong? |
| 49 | Gemini 1.5 Pro | paper | 69 | 2024 | Paper ↗ | Looks wrong? |
| 50 | MiMo V2.5 Pro | vendor | 68.5 | N/A | Code ↗ | Looks wrong? |
| 51 | Claude 3 Opus | paper | 68.5 | 2024 | Paper ↗ | Looks wrong? |
| 52 | Qwen3 235B A22B | vendor | 68.18 | N/A | Code ↗Source ↗ | Looks wrong? |
| 53 | Mistral Large Instruct 2411 | vendor | 67.94 | N/A | Code ↗Source ↗ | Looks wrong? |
| 54 | Hunyuan A13B Instruct | vendor | 67.3 | N/A | Code ↗Source ↗ | Looks wrong? |
| 55 | Mistral Large Instruct 2407 | vendor | 65.91 | N/A | Code ↗Source ↗ | Looks wrong? |
| 56 | DeepSeek V2.5 | vendor | 65.83 | N/A | Code ↗Source ↗ | Looks wrong? |
| 57 | Seed OSS 36B Base | vendor | 65.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 58 | NVIDIA Nemotron 3 Nano 30B A3B Base BF16 | vendor | 65.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 59 | DeepSeek V3 | vendor | 64.4 | N/A | Code ↗ | Looks wrong? |
| 60 | granite 4.1 30b | vendor | 64.09 | N/A | Code ↗ | Looks wrong? |
| 61 | GPT-4-Turbo | paper | 63.7 | 2024 | Paper ↗ | Looks wrong? |
| 62 | Qwen2.5 14B | vendor | 63.69 | N/A | Code ↗Source ↗ | Looks wrong? |
| 63 | Qwen3 30B A3B Base | vendor | 61.7 | N/A | Code ↗Source ↗ | Looks wrong? |
| 64 | Llama 3.1 405B | vendor | 61.6 | N/A | Code ↗Source ↗ | Looks wrong? |
| 65 | Nemotron H 56B Base 8K | vendor | 60.5 | N/A | Code ↗Source ↗ | Looks wrong? |
| 66 | Seed OSS 36B Base woSyn | vendor | 60.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 67 | Tencent Hunyuan Large | vendor | 60.2 | N/A | Code ↗Source ↗ | Looks wrong? |
| 68 | Mellum2 12B A2.5B Base Pretrain | vendor | 59.31 | N/A | Code ↗ | Looks wrong? |
| 69 | Mellum2 12B A2.5B Base | vendor | 59.31 | N/A | Code ↗ | Looks wrong? |
| 70 | Gemini 1.5 Flash | paper | 59.1 | 2024 | Paper ↗ | Looks wrong? |
| 71 | EXAONE 3.5 32B Instruct | vendor | 58.91 | N/A | Code ↗Source ↗ | Looks wrong? |
| 72 | MiMo 7B RL | vendor | 58.6 | N/A | Code ↗Source ↗ | Looks wrong? |
| 73 | internlm3 8b instruct | vendor | 57.6 | N/A | Code ↗Source ↗ | Looks wrong? |
| 74 | ERNIE 4.5 21B A3B Base PT | vendor | 56.7 | N/A | Code ↗Source ↗ | Looks wrong? |
| 75 | Llama 3 70B Instruct | paper | 56.2 | 2024 | Paper ↗ | Looks wrong? |
| 76 | granite 4.1 8b | vendor | 55.99 | N/A | Code ↗ | Looks wrong? |
| 77 | Phi 3 medium 4k instruct | vendor | 55.7 | N/A | Code ↗Source ↗ | Looks wrong? |
| 78 | DeepSeek V2 Chat | vendor | 54.81 | N/A | Code ↗Source ↗ | Looks wrong? |
| 79 | Mistral Small 24B Base 2501 | vendor | 54.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 80 | Phi 4 mini instruct | vendor | 52.8 | N/A | Code ↗Source ↗ | Looks wrong? |
| 81 | Meta Llama 3 70B | vendor | 52.78 | N/A | Code ↗Source ↗ | Looks wrong? |
| 82 | Llama 3.1 70B | vendor | 52.47 | N/A | Code ↗Source ↗ | Looks wrong? |
| 83 | Yi 1.5 34B Chat | vendor | 52.29 | N/A | Code ↗Source ↗ | Looks wrong? |
| 84 | Phi 3 medium 128k instruct | vendor | 51.91 | N/A | Code ↗Source ↗ | Looks wrong? |
| 85 | MAmmoTH2 8x7B Plus | vendor | 50.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 86 | Qwen1.5 110B | vendor | 49.93 | N/A | Code ↗Source ↗ | Looks wrong? |
| 87 | granite 4.1 3b | vendor | 49.83 | N/A | Code ↗ | Looks wrong? |
| 88 | AI21 Jamba Large 1.5 | vendor | 49.46 | N/A | Code ↗Source ↗ | Looks wrong? |
| 89 | Mistral Small Instruct 2409 | vendor | 48.4 | N/A | Code ↗Source ↗ | Looks wrong? |
| 90 | glm 4 9b | vendor | 47.92 | N/A | Code ↗Source ↗ | Looks wrong? |
| 91 | Phi 3.5 mini instruct | vendor | 47.87 | N/A | Code ↗Source ↗ | Looks wrong? |
| 92 | EXAONE 3.5 7.8B Instruct | vendor | 46.24 | N/A | Code ↗Source ↗ | Looks wrong? |
| 93 | Yi 1.5 9B Chat | vendor | 45.95 | N/A | Code ↗Source ↗ | Looks wrong? |
| 94 | Phi 3 mini 4k instruct | vendor | 45.66 | N/A | Code ↗Source ↗ | Looks wrong? |
| 95 | aya expanse 32b | vendor | 45.41 | N/A | Code ↗Source ↗ | Looks wrong? |
| 96 | gemma 2 9b | vendor | 45.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 97 | Qwen2.5 7B | vendor | 45 | N/A | Code ↗Source ↗ | Looks wrong? |
| 98 | Phi 3 mini 128k instruct | vendor | 43.86 | N/A | Code ↗Source ↗ | Looks wrong? |
| 99 | Qwen2.5 3B | vendor | 43.73 | N/A | Code ↗Source ↗ | Looks wrong? |
| 100 | MAmmoTH2 8B Plus | vendor | 43.35 | N/A | Code ↗Source ↗ | Looks wrong? |
| 101 | Yi 34B | vendor | 43.03 | N/A | Code ↗Source ↗ | Looks wrong? |
| 102 | Mathstral 7B v0.1 | vendor | 42 | N/A | Code ↗Source ↗ | Looks wrong? |
| 103 | MiMo 7B Base | vendor | 41.9 | N/A | Code ↗Source ↗ | Looks wrong? |
| 104 | DeepSeek Coder V2 Lite Instruct | vendor | 41.57 | N/A | Code ↗Source ↗ | Looks wrong? |
| 105 | Mixtral 8x7B v0.1 | vendor | 41.03 | N/A | Code ↗Source ↗ | Looks wrong? |
| 106 | Meta Llama 3 8B Instruct | vendor | 40.98 | N/A | Code ↗Source ↗ | Looks wrong? |
| 107 | MAmmoTH2 7B Plus | vendor | 40.85 | N/A | Code ↗Source ↗ | Looks wrong? |
| 108 | Qwen2 7B | vendor | 40.73 | N/A | Code ↗Source ↗ | Looks wrong? |
| 109 | Mistral Nemo Base 2407 | vendor | 39.77 | N/A | Code ↗Source ↗ | Looks wrong? |
| 110 | EXAONE 3.5 2.4B Instruct | vendor | 39.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 111 | Yi 1.5 6B Chat | vendor | 38.23 | N/A | Code ↗Source ↗ | Looks wrong? |
| 112 | Qwen1.5 14B Chat | vendor | 38.02 | N/A | Code ↗Source ↗ | Looks wrong? |
| 113 | Ministral 8B Instruct 2410 | vendor | 37.93 | N/A | Code ↗Source ↗ | Looks wrong? |
| 114 | c4ai command r v01 | vendor | 37.9 | N/A | Code ↗Source ↗ | Looks wrong? |
| 115 | internlm2 math plus 20b | vendor | 37.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 116 | LLaDA 8B Instruct | vendor | 37 | N/A | Code ↗Source ↗ | Looks wrong? |
| 117 | Llama 3 Smaug 8B | vendor | 36.93 | N/A | Code ↗Source ↗ | Looks wrong? |
| 118 | Llama 3.1 8B | vendor | 36.6 | N/A | Code ↗Source ↗ | Looks wrong? |
| 119 | Meta Llama 3 8B | vendor | 35.36 | N/A | Code ↗Source ↗ | Looks wrong? |
| 120 | deepseek math 7b instruct | vendor | 35.3 | N/A | Code ↗Source ↗ | Looks wrong? |
| 121 | DeepSeek Coder V2 Lite Base | vendor | 34.37 | N/A | Code ↗Source ↗ | Looks wrong? |
| 122 | aya expanse 8b | vendor | 33.74 | N/A | Code ↗Source ↗ | Looks wrong? |
| 123 | internlm2 math plus 7b | vendor | 33.5 | N/A | Code ↗Source ↗ | Looks wrong? |
| 124 | granite 3.1 8b base | vendor | 33.08 | N/A | Code ↗Source ↗ | Looks wrong? |
| 125 | Qwen2.5 1.5B | vendor | 32.1 | N/A | Code ↗Source ↗ | Looks wrong? |
| 126 | granite 3.0 8b base | vendor | 31.03 | N/A | Code ↗Source ↗ | Looks wrong? |
| 127 | Mistral 7B Instruct v0.2 | vendor | 30.84 | N/A | Code ↗Source ↗ | Looks wrong? |
| 128 | Mistral 7B v0.2 | vendor | 30.43 | N/A | Code ↗Source ↗ | Looks wrong? |
| 129 | Qwen1.5 7B Chat | vendor | 29.06 | N/A | Code ↗Source ↗ | Looks wrong? |
| 130 | Yi 6B Chat | vendor | 28.84 | N/A | Code ↗Source ↗ | Looks wrong? |
| 131 | Yi 6B | vendor | 26.51 | N/A | Code ↗Source ↗ | Looks wrong? |
| 132 | granite 3.1 2b base | vendor | 23.89 | N/A | Code ↗Source ↗ | Looks wrong? |
| 133 | llemma 7b | vendor | 23.45 | N/A | Code ↗Source ↗ | Looks wrong? |
| 134 | Qwen2 1.5B Instruct | vendor | 22.62 | N/A | Code ↗Source ↗ | Looks wrong? |
| 135 | Qwen2 1.5B | vendor | 22.56 | N/A | Code ↗Source ↗ | Looks wrong? |
| 136 | Llama 3.2 3B | vendor | 22.17 | N/A | Code ↗Source ↗ | Looks wrong? |
| 137 | granite 3.0 2b base | vendor | 21.72 | N/A | Code ↗Source ↗ | Looks wrong? |
| 138 | granite 3.1 3b a800m base | vendor | 20.39 | N/A | Code ↗Source ↗ | Looks wrong? |
| 139 | SmolLM2 1.7B | vendor | 18.31 | N/A | Code ↗Source ↗ | Looks wrong? |
| 140 | Qwen2 0.5B | vendor | 14.97 | N/A | Code ↗Source ↗ | Looks wrong? |
| 141 | Qwen2.5 0.5B | vendor | 14.92 | N/A | Code ↗Source ↗ | Looks wrong? |
| 142 | granite 3.1 1b a400m base | vendor | 12.34 | N/A | Code ↗Source ↗ | Looks wrong? |
| 143 | Llama 3.2 1B | vendor | 11.95 | N/A | Code ↗Source ↗ | Looks wrong? |
| 144 | SmolLM2 360M | vendor | 11.38 | N/A | Code ↗Source ↗ | Looks wrong? |
| 145 | SmolLM2 135M | vendor | 10.85 | N/A | Code ↗Source ↗ | Looks wrong? |
| 146 | Qwen2.5 VL 72B Instruct | vendor | 0.65 | N/A | Code ↗Source ↗ | Looks wrong? |