Web and desktop agents — AI systems that operate browsers and GUIs to complete real tasks — are benchmarked by WebArena, VisualWebArena, Mind2Web, and OSWorld. Current agents (GPT-4V + Playwright, Claude Computer Use) achieve 15-35% success on realistic web tasks, far below human performance. The core difficulty is grounding: mapping high-level instructions ("book a flight under $300") to pixel-level or DOM-level actions across unpredictable, dynamic interfaces. This is where multimodal understanding meets sequential decision-making, and progress here directly predicts when AI assistants can truly act on your behalf.
369 real computer tasks across Windows, macOS, and Ubuntu requiring GUI interaction. Tests agents operating full desktop apps like spreadsheets, image editors, and terminals. Much harder than web-only benchmarks.
Leading models on OSWorld.
| # | Model | success-rate | Year | Source |
|---|---|---|---|---|
| ★ | CoAct-1✓ | 60.8 | 2026 | paper ↗ |
| 2 | UI-TARS-2✓ | 47.5 | 2026 | paper ↗ |
| 3 | GTA1 (7B)✓ | 45.2 | 2026 | paper ↗ |
| 4 | UI-TARS-1.5✓ | 42.5 | 2026 | paper ↗ |
| 5 | Agent S2 (Gemini 2.5)✓ | 41.4 | 2026 | paper ↗ |
| 6 | OpenAI CUA (o1)✓ | 38.1 | 2026 | paper ↗ |
| 7 | Agent S2 (Claude 3.7)✓ | 34.5 | 2026 | paper ↗ |
| 8 | Claude 3.7 Sonnet✓ | 28.0 | 2026 | paper ↗ |
| 9 | UI-TARS-72B✓ | 24.6 | 2026 | paper ↗ |
| 10 | Claude Computer Use✓ | 22.0 | 2026 | paper ↗ |
Didn't find the model, metric, or dataset you needed? Tell us in one line. We read every message and reply within 48 hours.
2 datasets tracked for this task.
Still looking for something on Web & Desktop Agents? A missing model, a stale score, a benchmark we should cover — drop it here and we'll handle it.
Real humans read every message. We track what people are asking for and prioritize accordingly.