D1A on Gemma 4 E2B (JohnP1 d1a-e2b)v1.7 · 10 s rule

Data: JevBench v1.7.7, published 2026-10-26 · method v1.7 (10 s rule), measured under the rule

Capability 45.5 · rank #120 of 166 on the open-weights board

Last measured: 2026-10-26 · measurement release: v1.7.7

Apache-2.0 declared on the pinned Hugging Face card and metadata; no standalone LICENSE file in the pinned tree. The author GitHub code declares Apache-2.0. Base google/gemma-4-E2B and dataset obligations also apply.

Base model: undisclosed

intelligence

8.9

calibration

82.1

speed

93.3

cost

60.6

Composite (secondary)

0.8 · Composite rank #186

Cost and measurement conditions

$0.021 per 1,000 decisions (estimated)

ESTIMATE (*est.), no base key, explicit labelled pair: the frozen pricing_v15 module (sha256 aa2afcd5a58d0c4cc0e3d6867d590c70b6ac58e971410c27faf9628f97bcf8b7, re-freeze of 22 Oct 2026) has no market-listing decision for the card's base string google/gemma-4-E2B (reference() raises 'base model lacks an explicit market-listing decision', checked on this module), and an explicit reference cannot rescue a declared base, so this row carries NO base_model key and names the real base here instead. SAME-FAMILY REFERENCE, ONE SIZE UP (self-hosted open weights, no bookable listing for this base). pricing_v15.reference() RAISES on google/gemma-4-E2B and on google/gemma-4-e2b - the base is in none of the three frozen tables - and the published page carries no exact-rate row on this base either (the Gemma-4-E2B entries on the board are system-one-open, listed not_measured, and classone-gemma4-e2b, which itself publishes at this same labelled pair). The row therefore carries its own labelled estimate from the nearest reference the frozen 25 Sep snapshot itself contains, deepinfra:google/gemma-4-E4B-it at USD 0.02 per 1M input and USD 0.10 per 1M output - the SAME Gemma-4 effective-parameter family, one size LARGER, so it is both the nearest available reference and the direction that cannot flatter this row - x this row's own measured tokens. Applied at USD 0.02 per 1M input and USD 0.10 per 1M output (the pair is explicit; deepinfra:google/gemma-4-E4B-it USD 0.02/M input, USD 0.1/M output). Estimated, not charged, not a quote from the author or any host. Tokens as reported natively by the usage block of the author's server: 1,648,992 input and 173,752 output tokens across all 1,500 answered items. Output tokens are NATIVE and NONZERO: the server reports the token count of the serialized answer it returns (a probability answer, no generated explanation), so the zero-output convention for native-probability rows is NOT applied and output is priced as reported at the full output rate. The native input unit counts header, state and question separately without a system turn or chat template, after the author's truncation; native input usage equals our first-party offline count exactly (0 mismatches over 1,500 items). The 15 items answered on a truncated prompt are INCLUDED in the usage and the cost at their truncated size. Arithmetic over all 1,500 rows: USD 0.033570 per 1,000 decisions = 0.52x the unchanged USD 0.06459465517241379 cap; the common 1,485-item basis figure is computed by the scorer. COST FIGURES: USD 0.020549 per 1,000 decisions on the common 1,485-item cost basis that the score uses (the same items for every token-priced row: all items except the 15 very long ones that the Jev reference refused); separately, USD 0.033570 per 1,000 decisions over all 1,500 rows of the draw. The common-basis figure is 0.32x the official cost cap USD 0.064595 (all-rows figure 0.52x).

Measured p50 latency: 0.032 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)

Ranking and cap use: 0.21 s (est. incl. network (2x + 150 ms))

Self-hosted model: measured p50 0.032 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.21 s. Model authors can ask us to time a hosted endpoint instead (on request).

x2 + 0.15 s (assumption, not measured)

Measured 26 Oct 2026 on one rented NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB in Finland over loopback. The two models ran sequentially in fresh isolated containers with source, weights and inputs read-only, no credentials and networking disabled. GPU rental is separate from the estimated model tariff.

Measured latency over 1,500 decisions: p50 0.032 s · p95 0.035 s

15 decisions over 10 s (15 of them answered correctly, counted as wrong).

A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.

Published source