Tool-use, multi-step workflows, web agents. The frontier that demos best and produces the widest gap between benchmark scores and production reliability.
Agentic AI evolved from experimental prototypes to production enterprise systems in 2025. Major platforms from Anthropic, Google, Microsoft, and OpenAI achieved 50-70% success on real-world coding tasks, yet the gap between benchmark performance and production reliability remains substantial. Success depends less on raw model scale and more on orchestration, error handling, and human oversight.
Each task opens onto a leaderboard of its canonical benchmark, with the full submission history and dated scores. Tasks without an indexed result are listed elsewhere in the register; the table below is sorted by result count.
Leading scores for the headline benchmarks in this area, drawn from the registry. Shaded rows mark the top result per task; follow any row into the full leaderboard.
| # | Task | Benchmark | Leading model | Score |
|---|---|---|---|---|
| 01 | Task agents | AcademiClaw: agentic frontier tasks benchmark | Gemini 3.1 Pro | 2857.0 avg-tokens-per-task-k |
| 02 | SWE-bench | SWE-bench Verified — Agentic Leaderboard | Claude Mythos Preview | 93.90 resolve-rate |
| 03 | Autonomous Coding | Terminal-Bench 2.0 | Codex / GPT-5.5 | 82.0% accuracy |
| 04 | Tool Use | Tau2-Bench: Agentic Tool-Use Benchmark | Claude Opus 4.5 | 79.00 pass_rate |
| 05 | Web & Desktop Agents | WebArena: A Realistic Web Environment for Building Autonomous Agents | Agent-E (GPT-4o) | 73.00 success-rate |
| 06 | Time Horizon | METR Autonomy Evaluation: Time Horizon | Claude Opus 4 | 60.00 task-horizon-minutes |
| 07 | HCAST | Human-Calibrated Autonomy Software Tasks | Claude Opus 4 | 55.00 success-rate |
| 08 | Bioinformatics Agents | BixBench: a Comprehensive Benchmark for LLM-based Agents in Computational Biology | GPT-4o | 17.0% accuracy |
| 09 | RE-Bench | Research Engineering Benchmark | o3 | 0.380 normalized-score |
Gemini 3 Flash balances strong performance (competitive with Gemini 3 Pro on many tasks) with low cost and latency. Claude 3.5 Sonnet offers 49% SWE-bench with minimal scaffolding requirements. Reserve Gemini 3 Pro (76.2% SWE-bench) for genuinely complex tasks.
AutoGen excels at multi-agent collaboration with strong team coordination features. LangGraph provides explicit state management for complex workflows. Both offer production-grade observability and enterprise deployment patterns.
o3 achieved 87% on GPQA Diamond (exceeds PhD experts). o4-mini delivers 99.5% on AIME 2025 with tool use at fraction of o3's cost. Inference-time scaling enables variable compute allocation based on problem difficulty.
Open-weight models now within 1.7% of proprietary systems on benchmarks. Enable local deployment, avoid vendor lock-in, support custom fine-tuning. Llama 3.3 and Qwen 3 offer strong reasoning with full control over infrastructure.
Google ADK provides enterprise-grade infrastructure with tight Vertex AI integration. Anthropic's Model Context Protocol (MCP) offers universal tool connectivity across platforms. Both support governance, compliance, and security requirements for regulated industries.
Use Agent Memory Benchmark for comparable provider claims. Audrey's public benchmark report and raw artifacts are useful local deterministic regression/performance evidence, but not a CodeSOTA leaderboard score until an official AMB harness run exists.
Domain specialization (finance, healthcare, legal) justifies 8-12 week fine-tuning investment. Llama 3.3 provides strong foundation for customization. Mistral Large offers European data residency for GDPR compliance.
OpenAI SDK offers simplest implementation path with strong GPT integration. Claude provides excellent documentation and developer experience. Both enable fast iteration without framework complexity.
Route simple tasks to fast, cheap models. Escalate complex cases to premium models. This optimization balances cost and success rate for high-volume deployments where uniform premium model use proves economically infeasible.
Models achieving 70%+ on SWE-bench still hallucinate confidently in multi-step workflows. Academic benchmarks optimize single-task accuracy while ignoring cost, latency, error propagation, and security. Production success requires guardrails, observability, and human oversight that benchmarks don't measure.
o3 and similar reasoning models deliver impressive results but cost 5-10x standard models. For 80% of enterprise use cases, Gemini 3 Flash or Claude 3.5 Sonnet provide better cost-performance. Save reasoning models for genuinely hard problems, route simple tasks to efficient models.
LangChain, AutoGen, and CrewAI provide value but teams often succeed with minimal scaffolding. The hard parts are observability, guardrails, memory management, and human-in-loop workflows. Don't let framework complexity distract from production fundamentals.
Agent memory should be tested on recall, update/delete behavior, contradictions, valid-time beliefs, and evidence-backed preflight decisions. Local artifacts are useful regression evidence, but provider leaderboards need a shared harness such as Agent Memory Benchmark before scores are comparable.
Research shows human-AI collaboration beats pure automation by 69% despite being slower. For consequential decisions (contracts, customer communication, financial transactions), keep humans in approval loops. Speed optimization that sacrifices accuracy destroys business value.
Generic models adapted to specific contexts consistently underperform domain-specialized agents. Finance, healthcare, and legal applications justify 8-12 week fine-tuning investments. Half of enterprise AI will be domain-specific by 2028.
The benchmarks above come from the same Postgres registry that powers the wider Codesota index. Each task has exactly one canonical dataset. Each score carries a metric direction, a date and — where possible — a reproduction status.
When a score regresses, the prior record stays visible. When a benchmark is contested, we mark it rather than delete it. The goal is a register that argues in public.
Sibling area hubs, the unified task index and the methodology that binds them.