Claude Sonnet 5 (High)

AA-Briefcase published 2026-09-10 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

GDPval-AA v2 v2 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

Terminal-Bench v4.0 (AA) v4.0 → · Published board — Tests terminal-based work on the 66-task release using mini-SWE-agent v2.4.6.

τ²-Bench Airline (OpenRouter run) published 2026-09-15 → · Published board — Multi-turn service agents making tool calls under strict policy constraints. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

LMArena Agent Arena (overall net improvement) published 2026-09-26 → · Published board — Ranking of models as the orchestrator in LMArena’s Agent Mode, scored by the causal net improvement over the average model across behavioural signals from real sessions.

AA Agentic Index published 2026-09-18 → · Published board — Artificial Analysis publishes one Agentic Index per model, measured on its primary configuration and relayed by OpenRouter's Benchmarks API; the value is attached at family scope on the deterministic representative. Artificial Analysis publishes this board without a version number; we keep the date each result was retained.

SciCode (AA subproblems) v1.0.1 → · Published board — Tests scientific Python programming with scientist-annotated background information.

CursorBench 4.0 (Cursor) v4.0 → · Published board — Agent evaluation on ambiguous, multi-file tasks drawn from real Cursor sessions.

DeepSWE (Datacurve, via Epoch AI) published 2026-09-15 → · mini-SWE-agent — Pass rate of coding agents on original, long-horizon software engineering tasks, run by Datacurve with the mini-swe-agent harness.

FrontierCode 1.1 Main (Cognition) v1.1 → · Claude Code — Whether a maintainer would merge the agent's pull request, on tasks crafted by open-source maintainers and graded with tests, rubrics and verifiers.

SWE-rebench, issues 15 May – 1 Jul 2026 (Nebius) 2026-05-15..2026-07-01 → · SWE-rebench fixed ReAct scaffold, tool mode — A model resolves 111 fresh GitHub issues from 65 repositories, created between 15 May and 1 July 2026, inside one fixed minimal agent scaffold, run five times.

FrontierCode 1.1 Extended (Cognition) v1.1 → · Claude Code — Whether a maintainer would merge the agent's pull request, on the full 150-task FrontierCode set crafted by open-source maintainers and graded with tests, rubrics and verifiers.

LMArena WebDev (Code Arena, overall) published 2026-09-26 → · Published board — Community pairwise-vote ranking of models on front-end web development tasks, including agentic multi-step coding, published by LMArena as a Bradley-Terry rating.

DesignArena Web Apps (agentic) published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

DesignArena Full-Stack published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

CursorBench 4.0 cost per task (Cursor) v4.0 → · Published board — Cursor's published USD cost per task for each CursorBench 4.0 model-effort configuration.

FrontierCode 1.1 Main cost per rollout (Cognition) v1.1 → · Claude Code — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Main model and reasoning effort.

GPQA Diamond (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running GPQA Diamond for this configuration.

τ²-Bench Airline (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running τ²-Bench Airline for this configuration.

FrontierCode 1.1 Extended cost per rollout (Cognition) v1.1 → · Claude Code — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Extended model and reasoning effort.

LisanBench word chains, v0.2.0 (Lisan al Gaib) v0.2.0 → · Published board — A model builds the longest chain of English words it can, each differing from the last by one letter, with no repeats and only dictionary words, from 50 starting words; any broken rule ends the chain.

Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.

AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.

GDP.pdf (AA) published 2026-09-10 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.

GPQA Diamond (OpenRouter run) published 2026-09-15 → · Published board — Graduate-level science questions that resist retrieval and reward careful reasoning. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

AutomationBench-AA v1.0.6 → · Published board — Tests multi-app SaaS workflows through REST tools on a held-out AutomationBench split.

Chartography 100-tasks → · Published board — Can a model read the charts professionals rely on — Kaplan-Meier curves, candlesticks, contour maps, Sankey diagrams, Bode plots, wind roses — from the image alone, to an expert-set precision?