What multimodal models actually do, in 2026.

The word “multimodal” has expanded far beyond feeding a single image to a language model. In 2026 there are six distinguishable capability tracks, and conflating them is the main reason a model chosen for one job disappoints on another.

MMMU (Massive Multi-discipline Multimodal Understanding)

College-level reasoning across 30 subjects requiring both image understanding and domain knowledge

Examples · Art history analysis, circuit diagram solving, medical image interpretation

Status · Far from saturated. Human expert: ~88.6%. Best model: ~74.8%.

MathVista (Mathematical reasoning in Visual contexts)

Math problem solving from charts, geometry diagrams, scientific figures, and word problems with visual elements

Examples · Reading bar chart values and computing percentages, solving geometry from diagrams

Status · Active. Human: ~60% (surprisingly low). Models approaching human level.

RealWorldQA (Real World Question Answering)

Practical visual understanding from real-world photos — spatial reasoning, navigation, everyday comprehension

Examples · Reading street signs, estimating distances, understanding physical layouts

Status · Active. Tests practical intelligence that benchmarks often miss.

ChartQA (Chart Question Answering)

Extracting data and answering questions about charts and plots

Examples · Finding max values in bar charts, computing trends from line graphs, reading pie chart segments

Status · Approaching saturation. Best models at ~88-89%. Human: ~92%.

DocVQA (Document Visual Question Answering)

Extracting information from scanned documents, forms, receipts, and reports

Examples · Reading values from invoices, finding dates in contracts, parsing table entries

Status · Near saturation. Open-source models (Qwen2.5-VL) hit 96.4%. Human: ~98%.

Video-MME (Video Multi-Modal Evaluation)

Understanding video content across short (< 2min), medium (4-15min), and long (30-60min) clips

Examples · Summarizing events, temporal ordering, cause-effect reasoning across scenes

Status · Very active. Best model: ~75.2% (Gemini). Huge room for improvement.

The most useful thing a VLM paper rarely discusses in the abstract is how its model fails. Below are the seven recurring failure modes across every model in this guide — the severity marker tracks how often each one causes production regressions rather than benchmark noise.

high

Counting objects

Models consistently miscount objects in cluttered scenes. Ask "how many red cars?" in a parking lot photo and expect errors once count exceeds ~7.

high

Spatial relationships

Left/right, above/below, and relative position questions remain unreliable. Models may say object A is left of B when it is clearly right.

medium

Fine-grained text in images

Small text, watermarks, and text at angles gets misread or hallucinated. License plates, serial numbers, and distant signage are particularly problematic.

high

Temporal reasoning in video

Most VLMs sample frames rather than processing true video. They miss motion, speed, and cause-effect that happens between sampled frames.

medium

Multi-step visual reasoning

Chains of visual inference (A implies B, B implies C from image) degrade rapidly. Accuracy drops ~15% per reasoning step.

medium

Negation and absence

Asking "is there NOT a dog in this image?" or "what is missing from this scene?" triggers frequent errors. Models are biased toward confirming presence.

high

Hallucinated OCR

When text is partially visible or blurry, models confidently fabricate plausible-looking text rather than admitting uncertainty.

Text-to-image generation (DALL·E 3, Midjourney v7, Stable Diffusion 3.5, Flux) and vision-language understanding (the models on this page) are fundamentally different capabilities, despite sharing the multimodal label.

The convergence trend. Some models now merge the two: GPT-5 can generate and understand images; Gemini 2.5 Pro generates images natively. But the underlying architectures remain different, and specialisation still wins on benchmarks. A great generator is not a great understander, and vice versa.