DeepSeek V4 Pro 0813 (Max)

AA-Briefcase published 2026-09-10 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

GDPval-AA v2 v2 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

Terminal-Bench v2.1 (AA) v2.1 → · Published board — Tests terminal-based work on the 89-task verified refresh using Terminus 2.

Terminal-Bench v4.0 (AA) v4.0 → · Published board — Tests terminal-based work on the 66-task release using mini-SWE-agent v2.4.6.

τ²-Bench Airline (OpenRouter run) published 2026-09-15 → · Published board — Multi-turn service agents making tool calls under strict policy constraints. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

Finance Agent v2 (Vals Index v2) v2 → · Published board — Multi-step financial reasoning tasks.

HLAB — Harvey's Legal Agent Benchmark (Vals Index v2) v2 → · Published board — Long-horizon legal work product creation. The index column is Harvey's Legal Agent Benchmark's own published standalone score, reported as accuracy — not comparable with Artificial Analysis' Harvey LAB-AA row.

Terminal-Bench 2.1 (Vals Index v2) v2 → · Published board — Command-line interface problem solving; the page states that each index column is that benchmark's own published standalone score, so the runner is not asserted here.

Vals Index v2 (Vals AI) v2 → · Published board — GDP-weighted average of agentic model performance across finance, coding and legal tasks.

Terminal-Bench 4.0 published 2026-09-22 → · Published board — Terminal-Bench 4.0 results published by Vals AI.

Terminal-Bench Science published 2026-09-23 → · Published board — Terminal-Bench Science results published by Vals AI.

Tax Agent Bench published 2026-09-23 → · Published board — Tax Agent Bench results published by Vals AI.

Artificial Analysis Coding Agent Index v1.4 v1.4 → · Codex — The retained AA Coding Agent Index measures coding-agent systems using the earlier three-component implementation.

Artificial Analysis Coding Agent Index v1.5 v1.5 → · Codex — Measures coding-agent systems on DeepSWE v1.1, Terminal-Bench 4.0 and SWE-Atlas-QnA.

SciCode (AA subproblems) v1.0.1 → · Published board — Tests scientific Python programming with scientist-annotated background information.

Code Migration (Vals Index v2 subset) v2 → · Published board — Porting projects to another language, including COBOL modernization.

Vibe Code Bench (Vals Index v2) v2 → · Published board — End-to-end app-building tasks.

Vibe Code Bench 1-100 published 2026-09-22 → · Published board — Vibe Code Bench 1-100 results published by Vals AI.

Vibe Code Bench v1.1 v1.1 → · Published board — Vibe Code Bench v1.1 results published by Vals AI.

ProgramBench published 2026-09-23 → · Published board — ProgramBench results published by Vals AI.

Code Migration (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Code Migration results published by Vals AI, distinct from its Vals Index v2 subset.

AA Coding Index v4.x · frozen 2026-10-27 → · Published board — Artificial Analysis withdrew the Coding Index with Intelligence Index v5.0 (2026-10-28). These are the last v4.x values, frozen with their own date (2026-10-27).

GPQA Diamond (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running GPQA Diamond for this configuration.

τ²-Bench Airline (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running τ²-Bench Airline for this configuration.

Vals Index v2 cost per test (Vals AI) v2 → · Published board — Vals AI's published USD cost per test for each model on the Vals Index v2.

Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.

AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.

SimpleQA Verified (run by Epoch AI) published 2026-09-16 → · Published board — Short fact-seeking questions answered without tools, testing whether a model knows a fact rather than guessing; Google DeepMind's cleaned version of OpenAI's SimpleQA, run by Epoch AI.

Legal Research Bench (Vals Index v2) v2 → · Published board — Case and statute research with citation-backed answers.

MedScribe published 2026-09-22 → · Published board — MedScribe results published by Vals AI.

MedCode published 2026-09-22 → · Published board — MedCode results published by Vals AI.

GDP.pdf (AA) published 2026-09-10 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

MLCR-AA published 2026-09-10 → · Published board — Tests medical-record synthesis and reasoning across long, fragmented case documents.

Context Arena — MRCR v2 (8-needle) 8-needle → · Published board — The model must find and reproduce the right one of eight near-identical earlier replies hidden in a long conversation, averaged over context lengths from 8k to 128k tokens.

FrontierMath Tier 4 v2 (Epoch AI) v2 → · Published board — The hardest, research-level tier of Epoch AI's unpublished FrontierMath problems, run by Epoch AI on its private v2 set.

FrontierMath Tiers 1–3 v2 (Epoch AI) v2 → · Published board — Unpublished, expert-written mathematics problems from undergraduate to research level with automatically checkable answers, run by Epoch AI on its private v2 set.

ProofBench v1.1 v1.1 → · Published board — ProofBench v1.1 results published by Vals AI.

IOI published 2026-09-23 → · Published board — IOI results published by Vals AI.

ARC-AGI-1 v1 → · Published board — The first-generation ARC-AGI benchmark measuring passive fluid intelligence, with leaderboard score plotted against cost-per-task.

ARC-AGI-2 v2 → · Published board — The second-generation ARC-AGI benchmark of passive fluid intelligence, with leaderboard score plotted against cost-per-task.

Chess Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 novel chess positions generated programmatically by Epoch AI, each with a single Stockfish-verified best move; probes spatial reasoning and planning.

Mystery Game Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 mid-game positions of a well-known game whose identity Epoch deliberately keeps undisclosed, generated programmatically like Chess Puzzles.

AA Intelligence Index v5.0 · snapshot-2026-11-03 → · Published board — Artificial Analysis Intelligence Index v5.0. Models AA has not rescored on v5.0 are listed as estimates from their frozen v4.x value (isotonic fit over the models measured on both versions), never as measurements.

BullshitBench V1 (clear pushback) published 2026-09-10 → · Published board — Whether a model rejects the broken premise of 55 deliberately nonsensical prompts (V1 question set) instead of answering them confidently.

BullshitBench V2 (clear pushback) published 2026-09-10 → · Published board — Whether a model rejects the broken premise of 100 deliberately nonsensical prompts (V2 question set) instead of answering them confidently.

CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.

GPQA Diamond (AA) published 2026-09-10 → · Published board — Tests graduate-level biology, physics and chemistry knowledge on the Diamond subset.

GPQA Diamond (Epoch AI run) published 2026-09-18 → · Published board — Epoch AI's own inspect-ai runs of GPQA Diamond, the 198-question graduate-level science multiple-choice set.

GPQA Diamond (OpenRouter run) published 2026-09-15 → · Published board — Graduate-level science questions that resist retrieval and reward careful reasoning. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

AutomationBench-AA v1.0.6 → · Published board — Tests multi-app SaaS workflows through REST tools on a held-out AutomationBench split.

EnterpriseOps-Gym-AA published 2026-09-10 → · Published board — Tests enterprise workflows through MCP tools against resettable application environments.

τ³-Banking (AA) v1.0.1 → · Published board — Tests banking support agents that retrieve policies and change account state through tools.

Toolathlon-Verified (HKUST NLP) 2026-06-30 → · Published board — An agent completes 108 real-world tool-use tasks across 32 MCP servers and 7 local toolkits, and each task is graded by executing the resulting state against the task's ground truth.

Excel Modeling Benchmark (Vals Index v2) v2 → · Published board — Building and editing financial models in spreadsheets.

Excel Modeling Benchmark (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Excel Modeling Benchmark results published by Vals AI, distinct from its Vals Index v2 subset.