Grok 3

deprecated by benchmark source

xAI · released 2025-02-19 · 0 offers

Context 1M tokens

Providers

No provider publishes an API price for this model yet, so no cost can be modeled.

Composite

3 of 7 inputs · 2 from the model family6 radar axes: DesignArena's two boards share one

15.6

AA Coding v4.x —Coding Agent v1.4 —AA Intelligence (est.) 5.3AA Agentic —Epoch ECI 138.3Software ECI 139.2DesignArena —/—

Radar: percentile among all models measured on each input; a gap means not measured.

Benchmark sheet

17 of 319 registered benchmark versions · bars show the percentile among all models measured on each benchmark.

2 of 17 values are xAI's own claims (†), not independent measurements; a matching independent result replaces a claim as soon as one exists.

Composite attachments (used in the score, not counted as exact benchmarks):

Epoch ECI · attached
Software ECI · attached

Compare this model →

Agentic

Terminal-Bench Hard (AA)4711.4%2026-09-10
Terminal-Bench Hard (AA) pinned revision 74221fb → · Published board — Tests a pinned 44-task hard subset of terminal-based work using Terminus 2.

Instruction-following

IFBench (AA single-turn)5546.9%2026-09-10
IFBench (AA single-turn) published 2026-09-10 → · Published board — Tests precise single-turn instructions with deterministic rule checks.

Knowledge

Humanity's Last Exam (AA text-only)154.1%2026-09-10
Humanity's Last Exam (AA text-only) published 2026-09-10 → · Published board — Tests expert-level knowledge on AA's text-only Humanity's Last Exam subset.
AA-Omniscience Index52-33.22026-09-10
AA-Omniscience Index published 2026-09-10 → · Published board — Tests factual reliability while rewarding correct answers and penalizing hallucinations.
UGI NatInt8456.82026-09-10
UGI NatInt published 2026-09-10 → · Published board — A private-question evaluation of general knowledge and reasoning across textbook, popular-culture and world-model topics.

Long-context

AA-LCR v1.15258.0%2026-09-10
AA-LCR v1.1 v1.1 → · Published board — Tests reasoning across multiple long documents with corrected answer keys and grading.

Math

AIME 2025 (AA)5558.0%2026-09-10
AIME 2025 (AA) v2025 → · Published board — Advanced mathematical problem solving on AIME I and II 2025.

Reasoning

ARC-AGI-1developer's claim5.5%^{† self-reported by the developer}2026-09-10
ARC-AGI-1 v1 → · Published board — The first-generation ARC-AGI benchmark measuring passive fluid intelligence, with leaderboard score plotted against cost-per-task.
ARC-AGI-2developer's claim0.0%^{† self-reported by the developer}2026-09-10
ARC-AGI-2 v2 → · Published board — The second-generation ARC-AGI benchmark of passive fluid intelligence, with leaderboard score plotted against cost-per-task.
SimpleBench5936.1%2026-09-10
SimpleBench published 2026-09-10 → · Published board — A multiple-choice text benchmark of over 200 questions covering spatio-temporal reasoning, social intelligence, and linguistic adversarial robustness, on which a non-specialized human baseline outperforms every tested LLM.
SimpleBench5936.1%2026-09-26
SimpleBench published 2026-09-26 → · Published board — A multiple-choice text benchmark of over 200 questions covering spatio-temporal reasoning, social intelligence, and linguistic adversarial robustness, on which a non-specialized human baseline outperforms every tested LLM.

Science

CritPt (AA)240.0%2026-09-10
CritPt (AA) published 2026-09-10 → · Published board — Tests research-level physics reasoning with Python, symbolic and numerical answers.
GPQA Diamond (AA)4769.3%2026-09-10
GPQA Diamond (AA) published 2026-09-10 → · Published board — Tests graduate-level biology, physics and chemistry knowledge on the Diamond subset.

Tool-use

τ²-Bench Telecom (AA)5248.8%2026-09-10
τ²-Bench Telecom (AA) published 2026-09-10 → · Published board — Tests dual-control telecom agents that coordinate tool use with a simulated user.

Uncensored

UGI Willingness936.502026-09-10
UGI Willingness published 2026-09-10 → · Published board — A private-question evaluation of how far a model follows challenging instructions before refusing or deviating.
UGI9963.52026-09-10
UGI published 2026-09-10 → · Published board — A private-question suite combining sensitive-topic knowledge with willingness to follow controversial instructions.

Writing

UGI Writing8658.72026-09-10
UGI Writing published 2026-09-10 → · Published board — A maintainer-owned writing evaluation considering intelligence, style, repetition and length adherence, informed by human preferences.

Unusual results

1 threshold-crossing signal flagged · 13 eligible benchmark families

How flags are calculated

Heuristic screen, not statistical significance: benchmark families are correlated and source uncertainty is unknown. Peer evidence needs ≥ 20 independently measured matched configurations from ≥ 10 distinct model families. A flag needs a directed population z-score of magnitude ≥ 1.5 and a gap of ≥ 1.5 from the leave-one-benchmark-family-out mean z in the same direction, over ≥ 5 other benchmark families.

unusually strong UGI published 2026-09-10
Why
Observed: 63.47 score
Peer mean 35.1234 · peer sd 14.30074 · n 100 · families 84
Directed z 1.982 · baseline z 0.25 · gap 1.732 · profile n 12
63.47 scoremeasuredobserved 2026-09-10huggingface.co ↗
Evidence
Axis: UGI · published 2026-09-10 · Published board
Exact value: 63.47 score
Observed: 2026-09-10T22:04:13Z · publication date: not recorded
Observation id: public:8fb3398bf473dd54cbda1576
Source: https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard/raw/main/ugi-leaderboard-data.csv
Exact observation and full protocol ↗

Missing coverage · 302 benchmark versions

No result does not mean a zero, or that the model was never tested. Collection failures and disputed versions retain their distinct status.