Who leads the OK-VQA benchmark?

PaLI-X-55B currently leads OK-VQA with a score of 66.1 on accuracy.

What is the state-of-the-art score on OK-VQA?

The state-of-the-art result on OK-VQA is 66.1 (accuracy), achieved by PaLI-X-55B as of 2023.

How many models are tracked on OK-VQA?

Codesota tracks 5 models on OK-VQA.

When was the OK-VQA leaderboard last updated?

The OK-VQA leaderboard on Codesota includes results through 2023, with the earliest tracked result from 2022.

Codesota · Benchmark · OK-VQAHome/Leaderboards/Multimodal Media/Visual Question Answering/OK-VQA

Unknown

OK-VQA.

Name: OK-VQA Benchmark Results
Creator: Unknown
Published: 2022-01-01
License: https://creativecommons.org/licenses/by/4.0/

14,055 questions requiring outside knowledge to answer. Tests models that must consult external knowledge sources beyond visual content.

Paper ↗Leaderboard ↓Lineage

§ 01 · SOTA history

Year over year.

§ 02 · Leaderboard

Results by metric.

Found a wrong score or missing run?

Use row edits to send a sourced correction into moderation.

Add / edit result ↗Report issue ↗

accuracy

Accuracy is the reported evaluation metric for OK-VQA. Codesota tracks published model scores on this metric so readers can compare state-of-the-art results across sources and model families.

Higher is better

Trust tiers for accuracyverifiedpapervendorcommunityunverified

Muted rows were not state of the art when published — an earlier or same-year result already scored better.

Rank	Model	Trust	Score	Year	Links	Fix
01	PaLI-X-55B PaLI-X 55B fine-tuned on OK-VQA. 2023. Google Research.	verified	66.1	2023	Source ↗	Looks wrong?
02	PaLI-17B PaLI-17B fine-tuned on OK-VQA. ICLR 2023. Google Research.	verified	64.5	2022	Source ↗	Looks wrong?
03	GPT-4V GPT-4V zero-shot on OK-VQA test (commonsense knowledge subset). Nov 2023. OpenAI.	verified	64.28	2023	Source ↗	Looks wrong?
04	Flamingo-80B Flamingo-80B, 32-shot. OK-VQA test set. NeurIPS 2022. DeepMind.	verified	57.8	2022	Source ↗	Looks wrong?
05	BLIP-2 (FlanT5XXL) BLIP-2 with FlanT5XXL backbone. Zero-shot OK-VQA test. ICML 2023. Salesforce.	verified	44.7	2023	Source ↗	Looks wrong?