bison-1v1.7 · 10 s rule

Data: JevBench v1.8.19, published 2026-11-21 · method v1.7 (10 s rule), measured under the rule

Capability 85.5 · API offering, listed but not ranked on the API leaderboard

Added after the v1.8.10 ranking review; it is ranked at the next review.

Last measured: 2026-11-21 · measurement release: v1.8.19

MIT (modifications) and Apache-2.0 (base model)

Base model: google/gemma-4-31B-itmodel card

intelligence

78.1

calibration

93.0

speed

90.6

cost

52.5

Composite (secondary)

74.6 · Not ranked on this board

Cost and measurement conditions

$0.038 per 1,000 decisions (estimated)

Baseten Model APIs launch list price, USD 0.1/M input with free output (https://www.baseten.co/library/bison-1/, read 21 Nov 2026). Counted under the frozen I-1 launch-list-price rule, excluding promotional credits. This is above the google/gemma-4-31b-it base-model reference, USD 0.09/M input and USD 0.34/M output from the frozen OpenRouter snapshot, so the list price counts. The API reported one output token per item; output is free. The common 1,489-item basis uses 571,160 measured input tokens and yields USD 0.0383586299529886 per 1,000 decisions. The full 1,500-item run used 972,512 input tokens and yields USD 0.0648341333333333 per 1,000 decisions.

p50 latency: 0.3 s

none (API)

Baseten Model APIs /v1/systemone (inference.baseten.co); native probabilities; served model decision-models/baseten-bison-31b-latest.

Measured latency over 1,500 decisions: p50 0.251 s · p95 0.350 s

5 decisions over 10 s (5 of them answered without error, counted as wrong).

A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.

Breakdowns (not part of the score)

Chance-corrected competence (0 = chance, 100 = perfect) by category on open+sealed. Raw values, not equated; the same cells feed the compare view.

Capability by subject topic

subject topics: not labelled for this draw yet (top-up pending)

Use cases (TypeSafe categories)

Model routing86.5n=148
Legal & compliance89.3n=56
Customer support88.3n=66
E-commerce87.4n=62
Insurance claims86.4n=62
Risk assessment80.7n=48
Financial crime74.4n=63
Feature extraction80.3n=72
Lead generation97.5n=75
Recruiting87.8n=65
Knowledge graphs94.9n=67
LLM guardrails93.9n=71
Moderation91.3n=59
Code linting97.8n=69
Search & retrieval99.2n=67
Science89.0n=42
Advertising76.8n=48
Gaming78.9n=58
Demand forecasting53.8n=66

Cells need at least 30 answered items; † fewer than 30 (indicative only; drawn as hollow, lighter points on the compare radars).

Model and serving disclosure

Hosted API via Baseten Model APIs, served as decision-models/baseten-bison-31b-latest (a -latest alias), measured 21 Nov 2026; not tied to a Hugging Face revision. Reference Hugging Face HEAD at run time: baselabs/bison-1@bf8350b2. Baseten documentation lists 131,072 context tokens, while its management API reported 32,768. Five items of 77–82k input tokens received answers with full reported usage; reported usage is not proof that every token was read. Failed or unanswered requests count as wrong answers. Baseten retention statements are vendor claims and were not audited.

Published source