Codesota · Multimodal · Visual Question Answering · RealWorldQATasks/Multimodal/Visual Question Answering
Visual Question Answering · benchmark dataset · 2024 · EN

RealWorldQA Visual Question Answering.

Real-world image question-answering benchmark used in multimodal model reports to test practical visual understanding.

Paper ↗Submit a result ↵
§ 01 · Leaderboard

Best published scores.

23 results indexed across 1 metric. Shaded row marks current SOTA; ties broken by submission date.

Primary
accuracy · higher is better
accuracy· primary
23 rows
#ModelOrgSubmittedPaper / codeaccuracy
01Qwen3.6-35B-A3B—Apr 2026pwc-dump · code85.30
02Qwen3.5-122B-A10BOpenAlibabaFeb 2026pwc-dump · code85.10
03Qwen3.6-27B—Apr 2026pwc-dump · code84.10
04Qwen3.5-Omni-Plus—Apr 2026Qwen3.5-Omni Technical Report84.10
05Qwen3.5-35B-A3BOpenAlibabaFeb 2026pwc-dump · code84.10
06Qwen3.5-397B-A17BOpenAlibabaFeb 2026pwc-dump · code83.90
07Qwen3.5-27BOpenAlibabaFeb 2026pwc-dump · code83.70
08Qwen3-VL-235B-A22B-ThinkingQwenNov 2025Qwen3-VL Technical Report · code81.30
09Qwen3-VL-235B-A22B-InstructQwenNov 2025Qwen3-VL Technical Report · code79.20
10Qwen2-VL 72BOpenAlibabaSep 2024Qwen2-VL: Enhancing Vision-Language Model's Perception o… · code77.80
11LongCat-Flash-Omni—Oct 2025LongCat-Flash-Omni Technical Report · code76
12Qwen2.5-VL-72B—Feb 2025Qwen2.5-VL Technical Report · code75.70
13VideoLLaMA3 7B—Jan 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for … · code72.70
14Qwen3-VL-8B-InstructQwenNov 2025Qwen3-VL Technical Report · code71.50
15Qwen2-VL 7BAlibabaSep 2024Qwen2-VL: Enhancing Vision-Language Model's Perception o… · code70.10
16Kimi-VL-A3B-Thinking-2506—Apr 2025Kimi-VL Technical Report · code70
17BLIP3-o (8B)—May 2025BLIP3-o: A Family of Fully Open Unified Multimodal Model… · code69
18Grok-1.5V—Apr 2024pwc-dump68.70
19Kimi-VL-A3B-Instruct—Apr 2025Kimi-VL Technical Report · code68.10
20VideoLLaMA3 2B—Jan 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for … · code67.30
21ZAYA1-VL-8B—May 2026pwc-dump · code65
22MiniCPM-Llama3-V 2.5—Aug 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phone · code63.50
23Qwen2-VL-2B—Sep 2024Qwen2-VL: Enhancing Vision-Language Model's Perception o… · code62.90
Fig 2 · Rows sorted by score within each metric. Shaded row marks SOTA. Dates reflect model or paper release where available, otherwise the date Codesota accessed the source.
§ 03 · Progress

5 steps
of state of the art.

Each row below marks a model that broke the previous record on accuracy. Intermediate submissions are kept in the leaderboard above; only SOTA-setting entries are re-listed here.

Higher scores win. Each subsequent entry improved upon the previous best.

SOTA line · accuracy
  1. Apr 12, 2024Grok-1.5V68.70
  2. Sep 18, 2024Qwen2-VL 72BAlibaba77.80
  3. Nov 26, 2025Qwen3-VL-235B-A22B-ThinkingQwen81.30
  4. Feb 16, 2026Qwen3.5-122B-A10BAlibaba85.10
  5. Apr 21, 2026Qwen3.6-35B-A3B85.30
Fig 3 · SOTA-setting models only. 5 entries span Apr 2024 → Apr 2026.
§ 04 · Literature

9 papers
tied to this benchmark.

Every paper below corresponds to at least one row in the leaderboard above. Click through for the arXiv preprint and, when available, the reference implementation.

§ 06 · Contribute

Have a score that beats
this table?

Submit a checkpoint and a reproduction script. We will run it, publish the score, and — if it takes the top — annotate the step on the progress chart with your name.

Submit a result ↵Read submission guide
What a submission needs
  • 01A public checkpoint or API endpoint
  • 02A reproduction script with frozen commit + seed
  • 03Declared evaluation environment (Python, deps)
  • 04One row per metric declared by this dataset
  • 05A contact so we can follow up on discrepancies
RealWorldQA — Visual Question Answering | CodeSOTA