Claude Opus 5 (Max)

AA-AnalystAgent published 2026-09-10 → · Published board — Tests analyst tasks using agentic Python execution across fourteen domains.

AA-Briefcase published 2026-09-10 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

GDPval-AA v2 v2 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

Harvey LAB-AA published 2026-09-10 → · Published board — Tests legal-work deliverables across practice areas on Harvey's private task set. Artificial Analysis' run, graded by one LLM judge against task rubrics — not the same run or scale as Vals AI's HLAB row.

Terminal-Bench v2.1 (AA) v2.1 → · Published board — Tests terminal-based work on the 89-task verified refresh using Terminus 2.

Terminal-Bench v4.0 (AA) v4.0 → · Published board — Tests terminal-based work on the 66-task release using mini-SWE-agent v2.4.6.

ApprenticeBench API (NeoCognition) published 2026-09-14 → · Claude Code — An agent completes the same 100 sequential accounts-payable tasks in the Odoo ERP with diminishing mentoring, using dedicated application API calls instead of the screen interface.

ApprenticeBench CUA (NeoCognition) published 2026-09-14 → · Claude Code — A computer-use agent operates the Odoo ERP through its screens across 100 sequential accounts-payable tasks with diminishing mentoring.

EBR-bench (Earthborne Rangers, Epoch AI) published 2026-09-18 → · Published board — Learning-capability test: models repeatedly play the obscure campaign board game Earthborne Rangers with note-taking, and the score measures whether results improve across playthroughs.

τ^τ-bench (Hyper-τ), release v1 (Sierra) release-v1 → · Claude Code — A coding agent builds a working customer-service agent from realistic evidence (policies, transcripts, recordings, a client API), and is scored by how well that agent then serves simulated customers on 53 held-out tasks.

OSWorld 2.0, August 2026 task release (XLANG Lab) v2026.08.08 → · batch tool — A computer-use agent completes 108 long-horizon, real-world workflows across 31 self-hosted websites and desktop applications, each taking a person about 1.6 hours.

RSI-Exam 0.1 (aiming-lab) v0.1 → · Claude Code — An agent spends a long, budgeted run improving an inherited, working-but-weak executable research method or harness on a visible split, and the single artifact it submits is re-run once on a sealed hidden split across 88 executable research tasks in six domains.

Terminal-Bench v4.0 → · Claude Code — A benchmark to measure and evolve with the frontier of agent work, whose homepage leaderboard reports resolution rate on Terminal-Bench 4.0 tasks.

Finance Agent v2 (Vals Index v2) v2 → · Published board — Multi-step financial reasoning tasks.

HLAB — Harvey's Legal Agent Benchmark (Vals Index v2) v2 → · Published board — Long-horizon legal work product creation. The index column is Harvey's Legal Agent Benchmark's own published standalone score, reported as accuracy — not comparable with Artificial Analysis' Harvey LAB-AA row.

Vals Index v2 (Vals AI) v2 → · Published board — GDP-weighted average of agentic model performance across finance, coding and legal tasks.

LMArena Agent Arena (overall net improvement) published 2026-09-26 → · Published board — Ranking of models as the orchestrator in LMArena’s Agent Mode, scored by the causal net improvement over the average model across behavioural signals from real sessions.

Artificial Analysis Coding Agent Index v1.4 v1.4 → · Claude Code — The retained AA Coding Agent Index measures coding-agent systems using the earlier three-component implementation.

Artificial Analysis Coding Agent Index v1.5 v1.5 → · Claude Code — Measures coding-agent systems on DeepSWE v1.1, Terminal-Bench 4.0 and SWE-Atlas-QnA.

SciCode (AA subproblems) v1.0.1 → · Published board — Tests scientific Python programming with scientist-annotated background information.

CursorBench 4.0 (Cursor) v4.0 → · Published board — Agent evaluation on ambiguous, multi-file tasks drawn from real Cursor sessions.

DeepSWE (Datacurve, via Epoch AI) published 2026-09-15 → · mini-SWE-agent — Pass rate of coding agents on original, long-horizon software engineering tasks, run by Datacurve with the mini-swe-agent harness.

FrontierCode 1.1 Main (Cognition) v1.1 → · Claude Code — Whether a maintainer would merge the agent's pull request, on tasks crafted by open-source maintainers and graded with tests, rubrics and verifiers.

KernelBench-CUDA: Grid MinGRU SPS (RTX PRO 6000) rtx-pro-6000 → · Published board — Non-LLM RL simulation: maximise simulation steps per second for a grid world with a MinGRU agent (roofline anchored at 150M peak SPS); fusion optional.

KernelBench-CUDA: MegaQwen Decode (RTX PRO 6000) rtx-pro-6000 → · Published board — Improve the known MegaQwen CUDA megakernel geometry for decode-only token throughput at context lengths 2k–128k (roofline anchored on that decode-only tok/s ceiling); prefill is untimed.

SWE-bench Multilingual published 2026-09-10 → · Published board — A 300-instance SWE-bench variant with tasks from 42 repositories across 9 programming languages.

SWE-bench Multimodal published 2026-09-10 → · Published board — A 480-instance SWE-bench variant whose issue descriptions include visual elements.

Code Migration (Vals Index v2 subset) v2 → · Published board — Porting projects to another language, including COBOL modernization.

MLS-Bench-Lite 30-tasks → · Claude Code (max effort) — AI agents get a fixed ML research scaffold with strong baselines and must invent one algorithmic change — a loss, optimizer, sampler or training procedure — whose gain holds across settings, seeds, datasets and scales; Lite is the 30-task subset over all 12 research domains.

FrontierCode 1.1 Extended (Cognition) v1.1 → · Claude Code — Whether a maintainer would merge the agent's pull request, on the full 150-task FrontierCode set crafted by open-source maintainers and graded with tests, rubrics and verifiers.

LMArena WebDev (Code Arena, overall) published 2026-09-26 → · Published board — Community pairwise-vote ranking of models on front-end web development tasks, including agentic multi-step coding, published by LMArena as a Bradley-Terry rating.

AA Coding Index v4.x · frozen 2026-10-27 → · Published board — Artificial Analysis withdrew the Coding Index with Intelligence Index v5.0 (2026-10-28). These are the last v4.x values, frozen with their own date (2026-10-27).

ApprenticeBench API cost per task (NeoCognition) published 2026-09-14 → · Claude Code — ApprenticeBench's published USD cost per task for each API-board model-harness-effort configuration on the 100-task accounts-payable job.

ApprenticeBench CUA cost per task (NeoCognition) published 2026-09-14 → · Claude Code — ApprenticeBench's published USD cost per task for each CUA model-harness-effort configuration on the 100-task accounts-payable job.

CursorBench 4.0 cost per task (Cursor) v4.0 → · Published board — Cursor's published USD cost per task for each CursorBench 4.0 model-effort configuration.

FrontierCode 1.1 Main cost per rollout (Cognition) v1.1 → · Claude Code — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Main model and reasoning effort.

Vals Index v2 cost per test (Vals AI) v2 → · Published board — Vals AI's published USD cost per test for each model on the Vals Index v2.

FrontierCode 1.1 Extended cost per rollout (Cognition) v1.1 → · Claude Code — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Extended model and reasoning effort.

Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.

AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.

SimpleQA Verified (run by Epoch AI) published 2026-09-16 → · Published board — Short fact-seeking questions answered without tools, testing whether a model knows a fact rather than guessing; Google DeepMind's cleaned version of OpenAI's SimpleQA, run by Epoch AI.

Legal Research Bench (Vals Index v2) v2 → · Published board — Case and statute research with citation-backed answers.

GDP.pdf (AA) published 2026-09-10 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

MLCR-AA published 2026-09-10 → · Published board — Tests medical-record synthesis and reasoning across long, fragmented case documents.

Context Arena — MRCR v2 (8-needle) 8-needle → · Published board — The model must find and reproduce the right one of eight near-identical earlier replies hidden in a long conversation, averaged over context lengths from 8k to 128k tokens.

FrontierMath Tier 4 v2 (Epoch AI) v2 → · Published board — The hardest, research-level tier of Epoch AI's unpublished FrontierMath problems, run by Epoch AI on its private v2 set.

FrontierMath Tiers 1–3 v2 (Epoch AI) v2 → · Published board — Unpublished, expert-written mathematics problems from undergraduate to research level with automatically checkable answers, run by Epoch AI on its private v2 set.

ArXivMath 06/2026 (MathArena) 2026-06 → · Published board — Research-level math problems with a checkable final answer, drawn from arXiv papers submitted in June 2026; each edition is restricted to papers published within the last month to reduce training-data contamination.

BrokenArXiv 06/2026 (MathArena) 2026-06 → · Published board — Plausible but false proof statements taken from June 2026 arXiv papers; a model scores by refusing to prove them and saying the statement is false as written.

Chess Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 novel chess positions generated programmatically by Epoch AI, each with a single Stockfish-verified best move; probes spatial reasoning and planning.

LiveBench 2026-06-25 → · Published board — A periodically refreshed suite of objectively graded tasks with separate category scores and an overall category mean.

Mystery Game Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 mid-game positions of a well-known game whose identity Epoch deliberately keeps undisclosed, generated programmatically like Chess Puzzles.

CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.

GPQA Diamond (AA) published 2026-09-10 → · Published board — Tests graduate-level biology, physics and chemistry knowledge on the Diamond subset.

GPQA Diamond (Epoch AI run) published 2026-09-18 → · Published board — Epoch AI's own inspect-ai runs of GPQA Diamond, the 198-question graduate-level science multiple-choice set.

AutomationBench-AA v1.0.6 → · Published board — Tests multi-app SaaS workflows through REST tools on a held-out AutomationBench split.

EnterpriseOps-Gym-AA published 2026-09-10 → · Published board — Tests enterprise workflows through MCP tools against resettable application environments.

τ³-Banking (AA) v1.0.1 → · Published board — Tests banking support agents that retrieve policies and change account state through tools.

Excel Modeling Benchmark (Vals Index v2) v2 → · Published board — Building and editing financial models in spreadsheets.

MMMU Pro (AA) published 2026-09-10 → · Published board — Tests multimodal understanding using challenging ten-option questions.

Chartography 100-tasks → · Published board — Can a model read the charts professionals rely on — Kaplan-Meier curves, candlesticks, contour maps, Sankey diagrams, Bode plots, wind roses — from the image alone, to an expert-set precision?

GDP.pdf 100-tasks → · Published board — Can a model answer real professional questions about the PDFs work runs on — manuals, wiring diagrams, filings, forms — so that every rubric criterion an expert wrote is met?

Furniture Assembly Benchmark (Epoch AI) published 2026-09-26 → · Published board — Spatial-reasoning test by Epoch AI: given a photo of a partly or fully assembled IKEA product and its official manual, the model must say whether the build is correct and, if not, name the step with the mistake.