Gemini 3.8 Flash (High)

AA-Briefcase published 2026-09-10 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

GDPval-AA v2 v2 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

Terminal-Bench v2.1 (AA) v2.1 → · Published board — Tests terminal-based work on the 89-task verified refresh using Terminus 2.

Terminal-Bench v4.0 (AA) v4.0 → · Published board — Tests terminal-based work on the 66-task release using mini-SWE-agent v2.4.6.

ApprenticeBench API (NeoCognition) published 2026-09-14 → · Claude Code — An agent completes the same 100 sequential accounts-payable tasks in the Odoo ERP with diminishing mentoring, using dedicated application API calls instead of the screen interface.

RSI-Exam 0.1 (aiming-lab) v0.1 → · Antigravity CLI — An agent spends a long, budgeted run improving an inherited, working-but-weak executable research method or harness on a visible split, and the single artifact it submits is re-run once on a sealed hidden split across 88 executable research tasks in six domains.

Terminal-Bench v4.0 → · mini-SWE-agent — A benchmark to measure and evolve with the frontier of agent work, whose homepage leaderboard reports resolution rate on Terminal-Bench 4.0 tasks.

Finance Agent v2 (Vals Index v2) v2 → · Published board — Multi-step financial reasoning tasks.

HLAB — Harvey's Legal Agent Benchmark (Vals Index v2) v2 → · Published board — Long-horizon legal work product creation. The index column is Harvey's Legal Agent Benchmark's own published standalone score, reported as accuracy — not comparable with Artificial Analysis' Harvey LAB-AA row.

Terminal-Bench 2.1 (Vals Index v2) v2 → · Published board — Command-line interface problem solving; the page states that each index column is that benchmark's own published standalone score, so the runner is not asserted here.

Vals Index v2 (Vals AI) v2 → · Published board — GDP-weighted average of agentic model performance across finance, coding and legal tasks.

Scale SEAL: SWE-Bench Pro V2 HARD published 2026-09-22 → · Published board — Scale SEAL: SWE-Bench Pro V2 HARD results published by Scale AI.

CUA-bench published 2026-09-18 → · Published board — CUA-bench results published by Vals AI.

Time Horizon Index: KSP published 2026-09-14 → · Published board — Time Horizon Index: KSP results published by Vals AI.

Terminal-Bench 4.0 published 2026-09-22 → · Published board — Terminal-Bench 4.0 results published by Vals AI.

Terminal-Bench Science published 2026-09-23 → · Published board — Terminal-Bench Science results published by Vals AI.

Tax Agent Bench published 2026-09-23 → · Published board — Tax Agent Bench results published by Vals AI.

AA Agentic Index published 2026-09-18 → · Published board — Artificial Analysis publishes one Agentic Index per model, measured on its primary configuration and relayed by OpenRouter's Benchmarks API; the value is attached at family scope on the deterministic representative. Artificial Analysis publishes this board without a version number; we keep the date each result was retained.

Artificial Analysis Coding Agent Index v1.4 v1.4 → · Antigravity SDK v0.1.12 — The retained AA Coding Agent Index measures coding-agent systems using the earlier three-component implementation.

Artificial Analysis Coding Agent Index v1.4 v1.4 → · Opencode — The retained AA Coding Agent Index measures coding-agent systems using the earlier three-component implementation.

Artificial Analysis Coding Agent Index v1.5 v1.5 → · Antigravity SDK v0.1.12 — Measures coding-agent systems on DeepSWE v1.1, Terminal-Bench 4.0 and SWE-Atlas-QnA.

Artificial Analysis Coding Agent Index v2.0 v2.0 → · Antigravity SDK v0.1.12 — Measures coding-agent systems on FrontierCode v1.1 Main, Terminal-Bench 4.0 and SWE-Atlas-QnA-AA.

SciCode (AA subproblems) v1.0.1 → · Published board — Tests scientific Python programming with scientist-annotated background information.

CursorBench 4.0 (Cursor) v4.0 → · Published board — Agent evaluation on ambiguous, multi-file tasks drawn from real Cursor sessions.

DeepSWE (Datacurve, via Epoch AI) published 2026-09-15 → · mini-SWE-agent — Pass rate of coding agents on original, long-horizon software engineering tasks, run by Datacurve with the mini-swe-agent harness.

FrontierCode 1.1 Main (Cognition) v1.1 → · chisel — Whether a maintainer would merge the agent's pull request, on tasks crafted by open-source maintainers and graded with tests, rubrics and verifiers.

KernelBench-CUDA: DeepSeek NSA (RTX PRO 6000) rtx-pro-6000 → · Published board — Implement DeepSeek’s Native Sparse Attention block (block top-n routing + sparse attention) as a CUDA kernel over a frozen shape sweep.

KernelBench-CUDA: Grid MinGRU SPS (RTX PRO 6000) rtx-pro-6000 → · Published board — Non-LLM RL simulation: maximise simulation steps per second for a grid world with a MinGRU agent (roofline anchored at 150M peak SPS); fusion optional.

KernelBench-CUDA: MegaQwen Decode (RTX PRO 6000) rtx-pro-6000 → · Published board — Improve the known MegaQwen CUDA megakernel geometry for decode-only token throughput at context lengths 2k–128k (roofline anchored on that decode-only tok/s ceiling); prefill is untimed.

Code Migration (Vals Index v2 subset) v2 → · Published board — Porting projects to another language, including COBOL modernization.

Vibe Code Bench (Vals Index v2) v2 → · Published board — End-to-end app-building tasks.

WeirdML v3 v3 → · Published board — Agentic benchmark with 11 hand-made machine-learning tasks: a model must explore unfamiliar data, build and run analysis pipelines and produce results despite limited data, unspecified goals or very limited feedback.

Vibe Code Bench 1-100 published 2026-09-22 → · Published board — Vibe Code Bench 1-100 results published by Vals AI.

Vibe Code Bench v1.1 v1.1 → · Published board — Vibe Code Bench v1.1 results published by Vals AI.

FrontierCode 1.1 Extended (Cognition) v1.1 → · chisel — Whether a maintainer would merge the agent's pull request, on the full 150-task FrontierCode set crafted by open-source maintainers and graded with tests, rubrics and verifiers.

ProgramBench published 2026-09-23 → · Published board — ProgramBench results published by Vals AI.

Code Migration (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Code Migration results published by Vals AI, distinct from its Vals Index v2 subset.

CyberBench v1.1 v1.1 → · Published board — CyberBench v1.1 results published by Vals AI.

Scale SEAL: SWE-Bench Pro V2 Full published 2026-09-26 → · Published board — Scale AI’s refreshed public SWE-Bench Pro split (642 long-horizon tasks in 11 copyleft repositories) run under a locked protocol, scored by resolve rate.

AA Coding Index v4.x · frozen 2026-10-27 → · Published board — Artificial Analysis withdrew the Coding Index with Intelligence Index v5.0 (2026-10-28). These are the last v4.x values, frozen with their own date (2026-10-27).

DesignArena Web Apps (agentic) published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

DesignArena Full-Stack published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

ApprenticeBench API cost per task (NeoCognition) published 2026-09-14 → · Claude Code — ApprenticeBench's published USD cost per task for each API-board model-harness-effort configuration on the 100-task accounts-payable job.

CursorBench 4.0 cost per task (Cursor) v4.0 → · Published board — Cursor's published USD cost per task for each CursorBench 4.0 model-effort configuration.

FrontierCode 1.1 Main cost per rollout (Cognition) v1.1 → · chisel — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Main model and reasoning effort.

Vals Index v2 cost per test (Vals AI) v2 → · Published board — Vals AI's published USD cost per test for each model on the Vals Index v2.

FrontierCode 1.1 Extended cost per rollout (Cognition) v1.1 → · chisel — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Extended model and reasoning effort.

Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.

AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.

SimpleQA Verified (run by Epoch AI) published 2026-09-16 → · Published board — Short fact-seeking questions answered without tools, testing whether a model knows a fact rather than guessing; Google DeepMind's cleaned version of OpenAI's SimpleQA, run by Epoch AI.

Legal Research Bench (Vals Index v2) v2 → · Published board — Case and statute research with citation-backed answers.

Public Benefits Bench v1.1 v1.1 → · Published board — Public Benefits Bench v1.1 results published by Vals AI.

MedScribe published 2026-09-22 → · Published board — MedScribe results published by Vals AI.

MedCode published 2026-09-22 → · Published board — MedCode results published by Vals AI.

GDP.pdf (AA) published 2026-09-10 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

MLCR-AA published 2026-09-10 → · Published board — Tests medical-record synthesis and reasoning across long, fragmented case documents.

FrontierMath Tier 4 v2 (Epoch AI) v2 → · Published board — The hardest, research-level tier of Epoch AI's unpublished FrontierMath problems, run by Epoch AI on its private v2 set.

FrontierMath Tiers 1–3 v2 (Epoch AI) v2 → · Published board — Unpublished, expert-written mathematics problems from undergraduate to research level with automatically checkable answers, run by Epoch AI on its private v2 set.

ProofBench v1.1 v1.1 → · Published board — ProofBench v1.1 results published by Vals AI.

IOI published 2026-09-23 → · Published board — IOI results published by Vals AI.

SAGE published 2026-09-22 → · Published board — SAGE results published by Vals AI.

ARC-AGI-1 v1 → · Published board — The first-generation ARC-AGI benchmark measuring passive fluid intelligence, with leaderboard score plotted against cost-per-task.

ARC-AGI-2 v2 → · Published board — The second-generation ARC-AGI benchmark of passive fluid intelligence, with leaderboard score plotted against cost-per-task.

Chess Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 novel chess positions generated programmatically by Epoch AI, each with a single Stockfish-verified best move; probes spatial reasoning and planning.

LiveBench 2026-06-25 → · Published board — A periodically refreshed suite of objectively graded tasks with separate category scores and an overall category mean.

Mystery Game Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 mid-game positions of a well-known game whose identity Epoch deliberately keeps undisclosed, generated programmatically like Chess Puzzles.

AA Intelligence Index v5.0 · snapshot-2026-11-03 → · Published board — Artificial Analysis Intelligence Index v5.0. Models AA has not rescored on v5.0 are listed as estimates from their frozen v4.x value (isotonic fit over the models measured on both versions), never as measurements.

CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.

GPQA Diamond (AA) published 2026-09-10 → · Published board — Tests graduate-level biology, physics and chemistry knowledge on the Diamond subset.

GPQA Diamond (Epoch AI run) published 2026-09-18 → · Published board — Epoch AI's own inspect-ai runs of GPQA Diamond, the 198-question graduate-level science multiple-choice set.

BioMysteryBench published 2026-09-22 → · Published board — BioMysteryBench results published by Vals AI.

MysteryMechanism published 2026-09-23 → · Published board — MysteryMechanism results published by Vals AI.

AutomationBench-AA v1.0.6 → · Published board — Tests multi-app SaaS workflows through REST tools on a held-out AutomationBench split.

τ³-Banking (AA) v1.0.1 → · Published board — Tests banking support agents that retrieve policies and change account state through tools.

Excel Modeling Benchmark (Vals Index v2) v2 → · Published board — Building and editing financial models in spreadsheets.

Excel Modeling Benchmark (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Excel Modeling Benchmark results published by Vals AI, distinct from its Vals Index v2 subset.

MMMU Pro (AA) published 2026-09-10 → · Published board — Tests multimodal understanding using challenging ten-option questions.

Chartography 100-tasks → · Published board — Can a model read the charts professionals rely on — Kaplan-Meier curves, candlesticks, contour maps, Sankey diagrams, Bode plots, wind roses — from the image alone, to an expert-set precision?

GDP.pdf 100-tasks → · Published board — Can a model answer real professional questions about the PDFs work runs on — manuals, wiring diagrams, filings, forms — so that every rubric criterion an expert wrote is met?

Furniture Assembly Benchmark (Epoch AI) published 2026-09-26 → · Published board — Spatial-reasoning test by Epoch AI: given a photo of a partly or fully assembled IKEA product and its official manual, the model must say whether the build is correct and, if not, name the step with the mistake.