Codesota · Multimodal · Image-Text-to-Text · MMStarTasks/Multimodal/Image-Text-to-Text
Image-Text-to-Text · benchmark dataset · 2024 · EN

MMStar.

Vision-indispensable multimodal benchmark filtering text-solvable questions

Submit a result ↵
§ 01 · Leaderboard

Best published scores.

21 results indexed across 1 metric. Shaded row marks current SOTA; ties broken by submission date.

Primary
accuracy · higher is better
accuracy· primary
21 rows
#ModelOrgSubmittedPaper / codeaccuracy
01Qwen3.5-397B-A17BOpenAlibabaFeb 2026pwc-dump · code83.80
02Qwen3.5-122B-A10BOpenAlibabaFeb 2026pwc-dump · code82.90
03Qwen3.5-35B-A3BOpenAlibabaFeb 2026pwc-dump · code81.90
04Qwen3.6-27B—Apr 2026pwc-dump · code81.40
05Qwen3.5-27BOpenAlibabaFeb 2026pwc-dump · code81
06SenseNova-U1-A3B-MoTSenseTimeMay 2026SenseNova-U1: Unifying Multimodal Understanding and Gene… · code80.92
07Qwen3.6-35B-A3B—Apr 2026pwc-dump · code80.70
08Qwen3.5-Omni-Plus—Apr 2026Qwen3.5-Omni Technical Report79.40
09Qwen3-VL-235B-A22B-ThinkingQwenNov 2025Qwen3-VL Technical Report · code78.70
10Qwen3-VL-235B-A22B-InstructQwenNov 2025Qwen3-VL Technical Report · code78.40
11MiniCPM-o 4.5-Instruct—Apr 2026MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal … · code73.10
12Ovis2.5-9B—Aug 2025Ovis2.5 Technical Report · code72.40
13Qwen3-VL-8B-InstructQwenNov 2025Qwen3-VL Technical Report · code70.90
14Qwen2.5-VL-72B—Feb 2025Qwen2.5-VL Technical Report · code70.80
15Kimi-VL-A3B-Thinking-2506—Apr 2025Kimi-VL Technical Report · code70.40
16Infinity-Parser2-Pro—May 2026pwc-dump69.66
17MiniCPM-V 4.6-Thinking (16x)—May 2026pwc-dump68.50
18Qwen2-VL 72BOpenAlibabaSep 2024Qwen2-VL: Enhancing Vision-Language Model's Perception o… · code68.30
19Kimi-VL-A3B-Instruct—Apr 2025Kimi-VL Technical Report · code61.30
20Qwen2-VL 7BAlibabaSep 2024Qwen2-VL: Enhancing Vision-Language Model's Perception o… · code60.70
21Qwen2-VL-2B—Sep 2024Qwen2-VL: Enhancing Vision-Language Model's Perception o… · code48
Fig 2 · Rows sorted by score within each metric. Shaded row marks SOTA. Dates reflect model or paper release where available, otherwise the date Codesota accessed the source.
§ 03 · Progress

5 steps
of state of the art.

Each row below marks a model that broke the previous record on accuracy. Intermediate submissions are kept in the leaderboard above; only SOTA-setting entries are re-listed here.

Higher scores win. Each subsequent entry improved upon the previous best.

SOTA line · accuracy
  1. Sep 18, 2024Qwen2-VL 72BAlibaba68.30
  2. Feb 19, 2025Qwen2.5-VL-72B70.80
  3. Aug 15, 2025Ovis2.5-9B72.40
  4. Nov 26, 2025Qwen3-VL-235B-A22B-ThinkingQwen78.70
  5. Feb 16, 2026Qwen3.5-397B-A17BAlibaba83.80
Fig 3 · SOTA-setting models only. 5 entries span Sep 2024 → Feb 2026.
§ 04 · Literature

8 papers
tied to this benchmark.

Every paper below corresponds to at least one row in the leaderboard above. Click through for the arXiv preprint and, when available, the reference implementation.

§ 06 · Contribute

Have a score that beats
this table?

Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.

Submit a result ↵Read submission guide
What a submission needs
  • 01A public checkpoint or API endpoint
  • 02A reproduction script with frozen commit + seed
  • 03Declared evaluation environment (Python, deps)
  • 04One row per metric declared by this dataset
  • 05A contact so we can follow up on discrepancies