GPT-6 Astra (Max)

AA-AnalystAgent published 2026-09-10 → · Published board — Tests analyst tasks using agentic Python execution across fourteen domains.

AA-Briefcase v1.1 v1.1 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

AA-Briefcase published 2026-09-10 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

GDPval-AA v2 v2 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

GDPval-AA v2.1 v2.1 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

Terminal-Bench v2.1 (AA) v2.1 → · Published board — Tests terminal-based work on the 89-task verified refresh using Terminus 2.

Terminal-Bench v4.0 (AA) v4.0 → · Published board — Tests terminal-based work on the 66-task release using mini-SWE-agent v2.4.6.

Terminal-Bench v4.0 (AA, upstream timeouts) 4.0-upstream-timeouts → · Published board — Tests terminal-based work on the 66-task release using AA's mini-swe-agent harness with the tasks' own timeouts.

ApprenticeBench API (NeoCognition) published 2026-09-14 → · Codex — An agent completes the same 100 sequential accounts-payable tasks in the Odoo ERP with diminishing mentoring, using dedicated application API calls instead of the screen interface.

ApprenticeBench CUA (NeoCognition) published 2026-09-14 → · Codex — A computer-use agent operates the Odoo ERP through its screens across 100 sequential accounts-payable tasks with diminishing mentoring.

EBR-bench (Earthborne Rangers, Epoch AI) published 2026-09-18 → · Published board — Learning-capability test: models repeatedly play the obscure campaign board game Earthborne Rangers with note-taking, and the score measures whether results improve across playthroughs.

RSI-Exam 0.1 (aiming-lab) v0.1 → · Codex — An agent spends a long, budgeted run improving an inherited, working-but-weak executable research method or harness on a visible split, and the single artifact it submits is re-run once on a sealed hidden split across 88 executable research tasks in six domains.

Terminal-Bench v4.0 → · Codex — A benchmark to measure and evolve with the frontier of agent work, whose homepage leaderboard reports resolution rate on Terminal-Bench 4.0 tasks.

Finance Agent v2 (Vals Index v2) v2 → · Published board — Multi-step financial reasoning tasks.

HLAB — Harvey's Legal Agent Benchmark (Vals Index v2) v2 → · Published board — Long-horizon legal work product creation. The index column is Harvey's Legal Agent Benchmark's own published standalone score, reported as accuracy — not comparable with Artificial Analysis' Harvey LAB-AA row.

Terminal-Bench 2.1 (Vals Index v2) v2 → · Published board — Command-line interface problem solving; the page states that each index column is that benchmark's own published standalone score, so the runner is not asserted here.

Vals Index v2 (Vals AI) v2 → · Published board — GDP-weighted average of agentic model performance across finance, coding and legal tasks.

AutomationBench 1.0.6 v1.0.6 → · Vendor-reported by OpenAI — AutomationBench 1.0.6 score OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

Agents' Last Exam V1 v1 → · Vendor-reported by OpenAI — Agents' Last Exam V1 score OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

OSWorld 2.0 offline (v2026.08.08 release) v2026.08.08 → · Vendor-reported by OpenAI — OSWorld 2.0 offline partial reward OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

CUA-bench published 2026-09-18 → · Published board — CUA-bench results published by Vals AI.

SRE Bench published 2026-09-22 → · Published board — SRE Bench results published by Vals AI.

Terminal-Bench 4.0 published 2026-09-21 → · Published board — Terminal-Bench 4.0 results published by Vals AI.

Terminal-Bench Science published 2026-09-21 → · Published board — Terminal-Bench Science results published by Vals AI.

Time Horizon Index: KSP published 2026-09-14 → · Published board — Time Horizon Index: KSP results published by Vals AI.

Terminal-Bench 4.0 published 2026-09-22 → · Published board — Terminal-Bench 4.0 results published by Vals AI.

Terminal-Bench Science published 2026-09-23 → · Published board — Terminal-Bench Science results published by Vals AI.

Tax Agent Bench published 2026-09-23 → · Published board — Tax Agent Bench results published by Vals AI.

LMArena Agent Arena (overall net improvement) published 2026-09-26 → · Published board — Ranking of models as the orchestrator in LMArena’s Agent Mode, scored by the causal net improvement over the average model across behavioural signals from real sessions.

Artificial Analysis Coding Agent Index v1.4 v1.4 → · Codex — The retained AA Coding Agent Index measures coding-agent systems using the earlier three-component implementation.

Artificial Analysis Coding Agent Index v1.5 v1.5 → · Codex — Measures coding-agent systems on DeepSWE v1.1, Terminal-Bench 4.0 and SWE-Atlas-QnA.

Artificial Analysis Coding Agent Index v2.0 v2.0 → · Codex — Measures coding-agent systems on FrontierCode v1.1 Main, Terminal-Bench 4.0 and SWE-Atlas-QnA-AA.

SciCode (AA subproblems) v1.0.1 → · Published board — Tests scientific Python programming with scientist-annotated background information.

DeepSWE (Datacurve, via Epoch AI) published 2026-09-15 → · mini-SWE-agent — Pass rate of coding agents on original, long-horizon software engineering tasks, run by Datacurve with the mini-swe-agent harness.

FrontierCode 1.1 Main (Cognition) v1.1 → · Codex — Whether a maintainer would merge the agent's pull request, on tasks crafted by open-source maintainers and graded with tests, rubrics and verifiers.

Code Migration (Vals Index v2 subset) v2 → · Published board — Porting projects to another language, including COBOL modernization.

Vibe Code Bench (Vals Index v2) v2 → · Published board — End-to-end app-building tasks.

VulcanBench Frontier v4 v4 → · Codex — 23 behavioural-reconstruction tasks: the model must repair a replacement implementation of a legacy program until hidden tests confirm it reproduces the program’s real drift from its written spec; the combined score weights functional correctness 50%, lint/complexity 8.5%, security 8.5% and judged Code quality 33%.

DeepSWE v1.1 v1.1 → · Vendor-reported by OpenAI — DeepSWE v1.1 score OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

ProgramBench published 2026-09-21 → · Published board — ProgramBench results published by Vals AI.

Vibe Code Bench 1-100 published 2026-09-22 → · Published board — Vibe Code Bench 1-100 results published by Vals AI.

Vibe Code Bench v1.1 v1.1 → · Published board — Vibe Code Bench v1.1 results published by Vals AI.

Code Migration (standalone Vals board, 2026-09-21 snapshot) published 2026-09-21 → · Published board — Standalone Code Migration results published by Vals AI, distinct from its Vals Index v2 subset.

FrontierCode 1.1 Extended (Cognition) v1.1 → · Codex — Whether a maintainer would merge the agent's pull request, on the full 150-task FrontierCode set crafted by open-source maintainers and graded with tests, rubrics and verifiers.

ProgramBench published 2026-09-23 → · Published board — ProgramBench results published by Vals AI.

Code Migration (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Code Migration results published by Vals AI, distinct from its Vals Index v2 subset.

CyberBench v1.1 v1.1 → · Published board — CyberBench v1.1 results published by Vals AI.

LMArena WebDev (Code Arena, overall) published 2026-09-26 → · Codex harness — Community pairwise-vote ranking of models on front-end web development tasks, including agentic multi-step coding, published by LMArena as a Bradley-Terry rating.

AA Coding Index v4.x · frozen 2026-10-27 → · Published board — Artificial Analysis withdrew the Coding Index with Intelligence Index v5.0 (2026-10-28). These are the last v4.x values, frozen with their own date (2026-10-27).

DesignArena Web Apps (agentic) published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

ApprenticeBench API cost per task (NeoCognition) published 2026-09-14 → · Codex — ApprenticeBench's published USD cost per task for each API-board model-harness-effort configuration on the 100-task accounts-payable job.

ApprenticeBench CUA cost per task (NeoCognition) published 2026-09-14 → · Codex — ApprenticeBench's published USD cost per task for each CUA model-harness-effort configuration on the 100-task accounts-payable job.

FrontierCode 1.1 Main cost per rollout (Cognition) v1.1 → · Codex — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Main model and reasoning effort.

Vals Index v2 cost per test (Vals AI) v2 → · Published board — Vals AI's published USD cost per test for each model on the Vals Index v2.

AutomationBench 1.0.6 cost per task v1.0.6 → · Published board — USD cost per AutomationBench 1.0.6 task OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

Cost to run AA Intelligence Index (total) v4.3.2 → · Published board — Cost to run AA Intelligence Index (total) results published by Artificial Analysis.

Cost per AA Intelligence Index task v4.3.2 → · Published board — Cost per AA Intelligence Index task results published by Artificial Analysis.

FrontierCode 1.1 Extended cost per rollout (Cognition) v1.1 → · Codex — Cognition's published mean USD spend per rollout for each FrontierCode 1.1 Extended model and reasoning effort.

Agents' Last Exam V1 cost per task v1 → · Published board — USD cost per Agents' Last Exam V1 task OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

DeepSWE v1.1 cost per task v1.1 → · Published board — USD cost per DeepSWE v1.1 task OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

OSWorld 2.0 offline (v2026.08.08 release) cost per task v2026.08.08 → · Published board — USD cost per OSWorld 2.0 offline (v2026.08.08 release) task OpenAI reports for its own GPT-6 models in the 2026-09-22 GPT-6 Sol and Luna launch post.

Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.

AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.

SimpleQA Verified (run by Epoch AI) published 2026-09-16 → · Published board — Short fact-seeking questions answered without tools, testing whether a model knows a fact rather than guessing; Google DeepMind's cleaned version of OpenAI's SimpleQA, run by Epoch AI.

Legal Research Bench (Vals Index v2) v2 → · Published board — Case and statute research with citation-backed answers.

MedScribe published 2026-09-21 → · Published board — MedScribe results published by Vals AI.

MedScribe published 2026-09-22 → · Published board — MedScribe results published by Vals AI.

MedCode published 2026-09-22 → · Published board — MedCode results published by Vals AI.

GDP.pdf (AA) published 2026-09-10 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

GDP.pdf (AA) published 2026-09-21 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

MLCR-AA published 2026-09-10 → · Published board — Tests medical-record synthesis and reasoning across long, fragmented case documents.

FrontierMath Tier 4 v2 (Epoch AI) v2 → · Published board — The hardest, research-level tier of Epoch AI's unpublished FrontierMath problems, run by Epoch AI on its private v2 set.

FrontierMath Tiers 1–3 v2 (Epoch AI) v2 → · Published board — Unpublished, expert-written mathematics problems from undergraduate to research level with automatically checkable answers, run by Epoch AI on its private v2 set.

ArXivMath 06/2026 (MathArena) 2026-06 → · Published board — Research-level math problems with a checkable final answer, drawn from arXiv papers submitted in June 2026; each edition is restricted to papers published within the last month to reduce training-data contamination.

ArXivMath 08/2026 (MathArena) 2026-08 → · Published board — Research-level math problems with a checkable final answer, drawn from arXiv papers submitted in August 2026; each edition is restricted to papers published within the last month to reduce training-data contamination.

BrokenArXiv 06/2026 (MathArena) 2026-06 → · Published board — Plausible but false proof statements taken from June 2026 arXiv papers; a model scores by refusing to prove them and saying the statement is false as written.

BrokenArXiv 08/2026 (MathArena) 2026-08 → · Published board — Plausible but false statements taken from August 2026 arXiv papers; a model scores by refusing to prove them and saying the statement is false as written.

IOI published 2026-09-21 → · Published board — IOI results published by Vals AI.

ProofBench v1.1 v1.1 → · Published board — ProofBench v1.1 results published by Vals AI.

IOI published 2026-09-23 → · Published board — IOI results published by Vals AI.

SAGE published 2026-09-22 → · Published board — SAGE results published by Vals AI.

FrontierMath Erdős (Epoch AI) published 2026-09-26 → · Published board — 68 open problems posed or studied by Paul Erdős, formalised in Lean 4, where a model agent must produce a complete Lean proof or disproof that passes the Comparator checker.

LMArena Text (overall) published 2026-09-22 → · Published board — LMArena Text (overall) results published by LMArena.

LMArena Vision published 2026-09-22 → · Published board — LMArena Vision results published by LMArena.

BALROG (average progress) published 2026-09-22 → · Published board — BALROG (average progress) results published by BALROG.

ARC-AGI-1 v1 → · Published board — The first-generation ARC-AGI benchmark measuring passive fluid intelligence, with leaderboard score plotted against cost-per-task.

ARC-AGI-2 v2 → · Published board — The second-generation ARC-AGI benchmark of passive fluid intelligence, with leaderboard score plotted against cost-per-task.

Chess Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 novel chess positions generated programmatically by Epoch AI, each with a single Stockfish-verified best move; probes spatial reasoning and planning.

LiveBench 2026-06-25 → · Published board — A periodically refreshed suite of objectively graded tasks with separate category scores and an overall category mean.

Mystery Game Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 mid-game positions of a well-known game whose identity Epoch deliberately keeps undisclosed, generated programmatically like Chess Puzzles.

AA Intelligence Index v5.0 · snapshot-2026-11-03 → · Published board — Artificial Analysis Intelligence Index v5.0. Models AA has not rescored on v5.0 are listed as estimates from their frozen v4.x value (isotonic fit over the models measured on both versions), never as measurements.

BullshitBench V1 (clear pushback) published 2026-09-10 → · Published board — Whether a model rejects the broken premise of 55 deliberately nonsensical prompts (V1 question set) instead of answering them confidently.

BullshitBench V2 (clear pushback) published 2026-09-10 → · Published board — Whether a model rejects the broken premise of 100 deliberately nonsensical prompts (V2 question set) instead of answering them confidently.

CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.

GPQA Diamond (AA) published 2026-09-10 → · Published board — Tests graduate-level biology, physics and chemistry knowledge on the Diamond subset.

GPQA Diamond (Epoch AI run) published 2026-09-18 → · Published board — Epoch AI's own inspect-ai runs of GPQA Diamond, the 198-question graduate-level science multiple-choice set.

Scale SEAL: DrugDiscoveryBench published 2026-09-22 → · Codex — Scale SEAL: DrugDiscoveryBench results published by Scale AI.

Scale SEAL: DrugDiscoveryBench published 2026-09-22 → · mini-SWE-agent — Scale SEAL: DrugDiscoveryBench results published by Scale AI.

BioMysteryBench published 2026-09-21 → · Published board — BioMysteryBench results published by Vals AI.

MysteryMechanism published 2026-09-21 → · Published board — MysteryMechanism results published by Vals AI.

BioMysteryBench published 2026-09-22 → · Published board — BioMysteryBench results published by Vals AI.

MysteryMechanism published 2026-09-23 → · Published board — MysteryMechanism results published by Vals AI.

AutomationBench-AA v1.0.6 → · Published board — Tests multi-app SaaS workflows through REST tools on a held-out AutomationBench split.

τ³-Banking (AA) v1.0.1 → · Published board — Tests banking support agents that retrieve policies and change account state through tools.

Excel Modeling Benchmark (Vals Index v2) v2 → · Published board — Building and editing financial models in spreadsheets.

Excel Modeling Benchmark (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Excel Modeling Benchmark results published by Vals AI, distinct from its Vals Index v2 subset.

MMMU Pro (AA) published 2026-09-10 → · Published board — Tests multimodal understanding using challenging ten-option questions.

Chartography 100-tasks → · Published board — Can a model read the charts professionals rely on — Kaplan-Meier curves, candlesticks, contour maps, Sankey diagrams, Bode plots, wind roses — from the image alone, to an expert-set precision?

GDP.pdf 100-tasks → · Published board — Can a model answer real professional questions about the PDFs work runs on — manuals, wiring diagrams, filings, forms — so that every rubric criterion an expert wrote is met?

Furniture Assembly Benchmark (Epoch AI) published 2026-09-26 → · Published board — Spatial-reasoning test by Epoch AI: given a photo of a partly or fully assembled IKEA product and its official manual, the model must say whether the build is correct and, if not, name the step with the mistake.