Kimi K3 (Max)

AA-AnalystAgent published 2026-09-10 → · Published board — Tests analyst tasks using agentic Python execution across fourteen domains.

APEX-Agents-AA published 2026-09-10 → · Published board — Tests professional-service tasks that require agents to produce locally graded deliverables.

AA-Briefcase published 2026-09-10 → · Published board — Tests multi-week professional knowledge-work projects with linked tasks and large source collections.

GDPval-AA v2 v2 → · Published board — Tests professional knowledge-work deliverables across occupations using AA's Stirrup harness.

Harvey LAB-AA published 2026-09-10 → · Published board — Tests legal-work deliverables across practice areas on Harvey's private task set. Artificial Analysis' run, graded by one LLM judge against task rubrics — not the same run or scale as Vals AI's HLAB row.

ITBench-AA published 2026-09-10 → · Published board — Tests root-cause diagnosis from offline Kubernetes incident snapshots.

Terminal-Bench v2.1 (AA) v2.1 → · Published board — Tests terminal-based work on the 89-task verified refresh using Terminus 2.

Terminal-Bench v4.0 (AA) v4.0 → · Published board — Tests terminal-based work on the 66-task release using mini-SWE-agent v2.4.6.

ApprenticeBench API (NeoCognition) published 2026-09-14 → · Claude Code — An agent completes the same 100 sequential accounts-payable tasks in the Odoo ERP with diminishing mentoring, using dedicated application API calls instead of the screen interface.

ApprenticeBench CUA (NeoCognition) published 2026-09-14 → · Claude Code — A computer-use agent operates the Odoo ERP through its screens across 100 sequential accounts-payable tasks with diminishing mentoring.

τ²-Bench Airline (OpenRouter run) published 2026-09-15 → · Published board — Multi-turn service agents making tool calls under strict policy constraints. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

HLAB — Harvey's Legal Agent Benchmark (Vals Index v2) v2 → · Published board — Long-horizon legal work product creation. The index column is Harvey's Legal Agent Benchmark's own published standalone score, reported as accuracy — not comparable with Artificial Analysis' Harvey LAB-AA row.

Scale SEAL: SWE-Bench Pro V2 HARD published 2026-09-22 → · Published board — Scale SEAL: SWE-Bench Pro V2 HARD results published by Scale AI.

Time Horizon Index: KSP published 2026-09-14 → · Published board — Time Horizon Index: KSP results published by Vals AI.

Terminal-Bench 4.0 published 2026-09-22 → · Published board — Terminal-Bench 4.0 results published by Vals AI.

Terminal-Bench Science published 2026-09-23 → · Published board — Terminal-Bench Science results published by Vals AI.

Tax Agent Bench published 2026-09-23 → · Published board — Tax Agent Bench results published by Vals AI.

LMArena Agent Arena (overall net improvement) published 2026-09-26 → · Published board — Ranking of models as the orchestrator in LMArena’s Agent Mode, scored by the causal net improvement over the average model across behavioural signals from real sessions.

AA Agentic Index published 2026-09-18 → · Published board — Artificial Analysis publishes one Agentic Index per model, measured on its primary configuration and relayed by OpenRouter's Benchmarks API; the value is attached at family scope on the deterministic representative. Artificial Analysis publishes this board without a version number; we keep the date each result was retained.

Artificial Analysis Coding Agent Index v2.0 v2.0 → · Kimi Code CLI — Measures coding-agent systems on FrontierCode v1.1 Main, Terminal-Bench 4.0 and SWE-Atlas-QnA-AA.

SciCode (AA subproblems) v1.0.1 → · Published board — Tests scientific Python programming with scientist-annotated background information.

DeepSWE (Datacurve, via Epoch AI) published 2026-09-15 → · mini-SWE-agent — Pass rate of coding agents on original, long-horizon software engineering tasks, run by Datacurve with the mini-swe-agent harness.

FrontierSWE v2 v2 → · proximus — 34 hand-written real-world software tasks: every model is evaluated with the site’s own Proximus harness at its maximum reasoning effort, 5 trials per task under a 20-hour budget, scored by the site’s own review protocol; the headline value is mean@5 in percent with best@5/worst@5 bounds.

Code Migration (Vals Index v2 subset) v2 → · Published board — Porting projects to another language, including COBOL modernization.

MLS-Bench-Lite 30-tasks → · Kimi-Code (max) — AI agents get a fixed ML research scaffold with strong baselines and must invent one algorithmic change — a loss, optimizer, sampler or training procedure — whose gain holds across settings, seeds, datasets and scales; Lite is the 30-task subset over all 12 research domains.

Vibe Code Bench 1-100 published 2026-09-22 → · Published board — Vibe Code Bench 1-100 results published by Vals AI.

Code Migration (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Code Migration results published by Vals AI, distinct from its Vals Index v2 subset.

Scale SEAL: SWE-Bench Pro V2 Full published 2026-09-26 → · Published board — Scale AI’s refreshed public SWE-Bench Pro split (642 long-horizon tasks in 11 copyleft repositories) run under a locked protocol, scored by resolve rate.

LMArena WebDev (Code Arena, overall) published 2026-09-26 → · Published board — Community pairwise-vote ranking of models on front-end web development tasks, including agentic multi-step coding, published by LMArena as a Bradley-Terry rating.

AA Coding Index v4.x · frozen 2026-10-27 → · Published board — Artificial Analysis withdrew the Coding Index with Intelligence Index v5.0 (2026-10-28). These are the last v4.x values, frozen with their own date (2026-10-27).

DesignArena Web Apps (agentic) published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

DesignArena Full-Stack published 2026-11-03 → · Published board — Published Elo. Exact source attachment is retained; fewer than 200 battles excludes a row from radar normalization and anomaly peers.

ApprenticeBench API cost per task (NeoCognition) published 2026-09-14 → · Claude Code — ApprenticeBench's published USD cost per task for each API-board model-harness-effort configuration on the 100-task accounts-payable job.

ApprenticeBench CUA cost per task (NeoCognition) published 2026-09-14 → · Claude Code — ApprenticeBench's published USD cost per task for each CUA model-harness-effort configuration on the 100-task accounts-payable job.

GPQA Diamond (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running GPQA Diamond for this configuration.

τ²-Bench Airline (OpenRouter run) — measured cost per task published 2026-09-15 → · Published board — Mean USD OpenRouter actually spent per task while running τ²-Bench Airline for this configuration.

LisanBench word chains, v0.2.0 (Lisan al Gaib) v0.2.0 → · Published board — A model builds the longest chain of English words it can, each differing from the last by one letter, with no repeats and only dictionary words, from 50 starting words; any broken rule ends the chain.

Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.

AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.

SimpleQA Verified (run by Epoch AI) published 2026-09-16 → · Published board — Short fact-seeking questions answered without tools, testing whether a model knows a fact rather than guessing; Google DeepMind's cleaned version of OpenAI's SimpleQA, run by Epoch AI.

GDP.pdf (AA) published 2026-09-10 → · Published board — Tests professional reasoning over long PDFs with AA document preparation and grading.

AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

MLCR-AA published 2026-09-10 → · Published board — Tests medical-record synthesis and reasoning across long, fragmented case documents.

Context Arena — MRCR v2 (8-needle) 8-needle → · Published board — The model must find and reproduce the right one of eight near-identical earlier replies hidden in a long conversation, averaged over context lengths from 8k to 128k tokens.

FrontierMath Tier 4 v2 (Epoch AI) v2 → · Published board — The hardest, research-level tier of Epoch AI's unpublished FrontierMath problems, run by Epoch AI on its private v2 set.

FrontierMath Tiers 1–3 v2 (Epoch AI) v2 → · Published board — Unpublished, expert-written mathematics problems from undergraduate to research level with automatically checkable answers, run by Epoch AI on its private v2 set.

IOI published 2026-09-23 → · Published board — IOI results published by Vals AI.

ARC-AGI-1 v1 → · Published board — The first-generation ARC-AGI benchmark measuring passive fluid intelligence, with leaderboard score plotted against cost-per-task.

ARC-AGI-2 v2 → · Published board — The second-generation ARC-AGI benchmark of passive fluid intelligence, with leaderboard score plotted against cost-per-task.

Chess Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 novel chess positions generated programmatically by Epoch AI, each with a single Stockfish-verified best move; probes spatial reasoning and planning.

Mystery Game Puzzles (Epoch AI) published 2026-09-18 → · Published board — Best-move selection on 100 mid-game positions of a well-known game whose identity Epoch deliberately keeps undisclosed, generated programmatically like Chess Puzzles.

SimpleBench published 2026-09-10 → · Published board — A multiple-choice text benchmark of over 200 questions covering spatio-temporal reasoning, social intelligence, and linguistic adversarial robustness, on which a non-specialized human baseline outperforms every tested LLM.

SimpleBench published 2026-09-26 → · Published board — A multiple-choice text benchmark of over 200 questions covering spatio-temporal reasoning, social intelligence, and linguistic adversarial robustness, on which a non-specialized human baseline outperforms every tested LLM.

CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.

GPQA Diamond (AA) published 2026-09-10 → · Published board — Tests graduate-level biology, physics and chemistry knowledge on the Diamond subset.

GPQA Diamond (Epoch AI run) published 2026-09-18 → · Published board — Epoch AI's own inspect-ai runs of GPQA Diamond, the 198-question graduate-level science multiple-choice set.

GPQA Diamond (OpenRouter run) published 2026-09-15 → · Published board — Graduate-level science questions that resist retrieval and reward careful reasoning. (OpenRouter's own reproducible run, as published on openrouter.ai/benchmarks.)

BioMysteryBench published 2026-09-22 → · Published board — BioMysteryBench results published by Vals AI.

AutomationBench-AA v1.0.6 → · Published board — Tests multi-app SaaS workflows through REST tools on a held-out AutomationBench split.

EnterpriseOps-Gym-AA published 2026-09-10 → · Published board — Tests enterprise workflows through MCP tools against resettable application environments.

τ³-Banking (AA) v1.0.1 → · Published board — Tests banking support agents that retrieve policies and change account state through tools.

MCP Atlas (Scale AI) published 2026-09-21 → · Published board — How reliably a model completes realistic multi-step workflows over real Model Context Protocol servers: discovering the right tool in a noisy tool menu, calling it with correct parameters, recovering from errors and synthesising the results into an accurate final answer.

Toolathlon-Verified (HKUST NLP) 2026-06-30 → · Published board — An agent completes 108 real-world tool-use tasks across 32 MCP servers and 7 local toolkits, and each task is graded by executing the resulting state against the task's ground truth.

Excel Modeling Benchmark (Vals Index v2) v2 → · Published board — Building and editing financial models in spreadsheets.

MCPMark Verified verified → · Published board — AI agents complete realistic multi-step tasks through Model Context Protocol servers for Notion, GitHub, a filesystem, Postgres and a Playwright browser, each checked by the task's own verification script; Verified is the maintainers' version-pinned, stabilized 127-task set.

Excel Modeling Benchmark (standalone Vals board, 2026-09-22 snapshot) published 2026-09-22 → · Published board — Standalone Excel Modeling Benchmark results published by Vals AI, distinct from its Vals Index v2 subset.

MMMU Pro (AA) published 2026-09-10 → · Published board — Tests multimodal understanding using challenging ten-option questions.

Chartography 100-tasks → · Published board — Can a model read the charts professionals rely on — Kaplan-Meier curves, candlesticks, contour maps, Sankey diagrams, Bode plots, wind roses — from the image alone, to an expert-set precision?

GDP.pdf 100-tasks → · Published board — Can a model answer real professional questions about the PDFs work runs on — manuals, wiring diagrams, filings, forms — so that every rubric criterion an expert wrote is met?

Furniture Assembly Benchmark (Epoch AI) published 2026-09-26 → · Published board — Spatial-reasoning test by Epoch AI: given a photo of a partly or fully assembled IKEA product and its official manual, the model must say whether the build is correct and, if not, name the step with the mistake.