MiniMax-M3

AA-AnalystAgent published 2026-09-10 → · Published board — Tests analyst tasks using agentic Python execution across fourteen domains.

AA-Briefcase published 2026-09-10 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

GDPval-AA v2 v2 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

Harvey LAB-AA published 2026-09-10 → · Published board — Tests legal-work deliverables across practice areas on Harvey's private task set. Artificial Analysis' run, graded by one LLM judge against task rubrics — not the same run or scale as Vals AI's HLAB row.

Terminal-Bench Hard (AA) pinned revision 74221fb → · Published board — Tests a pinned 44-task hard subset of terminal-based work using Terminus 2.

Terminal-Bench v2.1 (AA) v2.1 → · Published board — Tests terminal-based work on the 89-task verified refresh using Terminus 2.

Terminal-Bench v4.0 (AA) v4.0 → · Published board — Tests terminal-based work on the 66-task release using mini-SWE-agent v2.4.6.

Long-Horizon Terminal-Bench v1.0 → · Terminus-2 — An agent works in a terminal for up to 90 minutes per task on 46 long jobs — building systems, migrating frameworks, playing games move by move — and earns partial credit for real progress.

τ²-Bench Airline (OpenRouter run) published 2026-09-15 → · Published board — Multi-turn service agents making tool calls under strict policy constraints. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

OSWorld 2.0, June 2026 task release (XLANG Lab) v2026.06.24 → · standard — A computer-use agent completes 108 long-horizon, real-world workflows across 31 self-hosted websites and desktop applications, each taking a person about 1.6 hours.

Finance Agent v2 (Vals Index v2) v2 → · Published board — Multi-step financial reasoning tasks.

HLAB — Harvey's Legal Agent Benchmark (Vals Index v2) v2 → · Published board — Long-horizon legal work product creation. The index column is Harvey's Legal Agent Benchmark's own published standalone score, reported as accuracy — not comparable with Artificial Analysis' Harvey LAB-AA row.

Terminal-Bench 2.1 (Vals Index v2) v2 → · Published board — Command-line interface problem solving; the page states that each index column is that benchmark's own published standalone score, so the runner is not asserted here.

Vals Index v2 (Vals AI) v2 → · Published board — GDP-weighted average of agentic model performance across finance, coding and legal tasks.

AA Agentic Index published 2026-09-18 → · Published board — Artificial Analysis publishes one Agentic Index per model, measured on its primary configuration and relayed by OpenRouter's Benchmarks API; the value is attached at family scope on the deterministic representative. Artificial Analysis publishes this board without a version number; we keep the date each result was retained.

SciCode (AA subproblems) v1.0.1 → · Published board — Tests scientific Python programming with scientist-annotated background information.

FrontierCode 1.1 Main (Cognition) v1.1 → · mini-SWE-agent — Whether a maintainer would merge the agent's pull request, on tasks crafted by open-source maintainers and graded with tests, rubrics and verifiers.

React Native Evals 91-evals → · Published board — Coding agents build React Native app features — navigation, animation, async state, lists, native APIs and Skia graphics — and an LLM judge checks the generated code against each task's written requirements.

SWE-rebench, issues 15 May – 1 Jul 2026 (Nebius) 2026-05-15..2026-07-01 → · SWE-rebench fixed ReAct scaffold, tool mode — A model resolves 111 fresh GitHub issues from 65 repositories, created between 15 May and 1 July 2026, inside one fixed minimal agent scaffold, run five times.

Code Migration (Vals Index v2 subset) v2 → · Published board — Porting projects to another language, including COBOL modernization.

Vibe Code Bench (Vals Index v2) v2 → · Published board — End-to-end app-building tasks.

FrontierCode 1.1 Extended (Cognition) v1.1 → · mini-SWE-agent — Whether a maintainer would merge the agent's pull request, on the full 150-task FrontierCode set crafted by open-source maintainers and graded with tests, rubrics and verifiers.

AA Coding Index v4.x · frozen 2026-10-27 → · Published board — Artificial Analysis withdrew the Coding Index with Intelligence Index v5.0 (2026-10-28). These are the last v4.x values, frozen with their own date (2026-10-27).

DesignArena Web Apps (agentic) published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

DesignArena Full-Stack published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

FrontierCode 1.1 Main cost per rollout (Cognition) v1.1 → · mini-SWE-agent — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Main model and reasoning effort.

GPQA Diamond (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running GPQA Diamond for this configuration.

τ²-Bench Airline (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running τ²-Bench Airline for this configuration.

Vals Index v2 cost per test (Vals AI) v2 → · Published board — Vals AI's published USD cost per test for each model on the Vals Index v2.

FrontierCode 1.1 Extended cost per rollout (Cognition) v1.1 → · mini-SWE-agent — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Extended model and reasoning effort.

IFBench (AA single-turn) published 2026-09-10 → · Published board — Tests precise single-turn instructions with deterministic rule checks.

Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.

AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.

Legal Research Bench (Vals Index v2) v2 → · Published board — Case and statute research with citation-backed answers.

GDP.pdf (AA) published 2026-09-10 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

MLCR-AA published 2026-09-10 → · Published board — Tests medical-record synthesis and reasoning across long, fragmented case documents.

Context Arena — MRCR v2 (8-needle) 8-needle → · Published board — The model must find and reproduce the right one of eight near-identical earlier replies hidden in a long conversation, averaged over context lengths from 8k to 128k tokens.

Chess Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 novel chess positions generated programmatically by Epoch AI, each with a single Stockfish-verified best move; probes spatial reasoning and planning.

LiveBench 2026-06-25 → · Published board — A periodically refreshed suite of objectively graded tasks with separate category scores and an overall category mean.

Mystery Game Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 mid-game positions of a well-known game whose identity Epoch deliberately keeps undisclosed, generated programmatically like Chess Puzzles.

AA Intelligence Index v5.0 · snapshot-2026-11-03 → · Published board — Artificial Analysis Intelligence Index v5.0. Models AA has not rescored on v5.0 are listed as estimates from their frozen v4.x value (isotonic fit over the models measured on both versions), never as measurements.

EQ-Bench 4 v4 → · Published board — Measures active emotional and social intelligence of LLMs in multi-turn roleplay chats with simulated personas, ranked by an LLM-judge panel Elo.

CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.

GPQA Diamond (AA) published 2026-09-10 → · Published board — Tests graduate-level biology, physics and chemistry knowledge on the Diamond subset.

GPQA Diamond (Epoch AI run) published 2026-09-18 → · Published board — Epoch AI's own inspect-ai runs of GPQA Diamond, the 198-question graduate-level science multiple-choice set.

GPQA Diamond (OpenRouter run) published 2026-09-15 → · Published board — Graduate-level science questions that resist retrieval and reward careful reasoning. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

ResearchClawBench 40-tasks → · ResearchHarness — An AI agent gets the raw data and related literature behind a published scientific paper — the paper itself hidden — and must redo the research end to end, from analysis to a written report, across 40 tasks in 10 scientific domains.

AutomationBench-AA v1.0.6 → · Published board — Tests multi-app SaaS workflows through REST tools on a held-out AutomationBench split.

EnterpriseOps-Gym-AA published 2026-09-10 → · Published board — Tests enterprise workflows through MCP tools against resettable application environments.

τ²-Bench Telecom (AA) published 2026-09-10 → · Published board — Tests dual-control telecom agents that coordinate tool use with a simulated user.

τ³-Banking (AA) v1.0.1 → · Published board — Tests banking support agents that retrieve policies and change account state through tools.

Excel Modeling Benchmark (Vals Index v2) v2 → · Published board — Building and editing financial models in spreadsheets.

MMMU Pro (AA) published 2026-09-10 → · Published board — Tests multimodal understanding using challenging ten-option questions.

LLM Creative Story-Writing Benchmark published 2026-09-10 → · Published board — Pairwise comparison of short stories written to the same constrained creative briefs, with LLM evaluator choices combined into a relative comparison score.