Decisio 12B tuned commit (variant of the Decisio family)v1.7 · 10 s rule

Data: JevBench v1.7.6, published 2026-10-25 · method v1.7 (10 s rule), measured under the rule

Capability 75.7 · rank #17 of 164 on the open-weights board

Last measured: 2026-10-25 · measurement release: v1.7.6

Apache-2.0 per the model card, LICENSE and NOTICE on the weights and the author server; base google/gemma-4-12B-it public, ungated, under its own terms

Base model: undisclosed

intelligence

62.8

calibration

88.6

speed

90.3

cost

58.5

Composite (secondary)

72.2 · Composite rank #5

Cost and measurement conditions

$0.024 per 1,000 decisions (estimated)

ESTIMATE (*est.), base-model market reference, explicit reference with NO base key: the frozen pricing_v15 module (sha256 aa2afcd5a58d0c4cc0e3d6867d590c70b6ac58e971410c27faf9628f97bcf8b7, re-freeze of 22 Oct 2026) has no listing for the card's base string google/gemma-4-12B-it (it is in the module's UNLISTED_BASE_MODELS set, so there is no exact rate and no floor; reference() returns nothing for it, checked on this module). The reference used is therefore google/gemma-3-12b-it USD 0.05/M input, USD 0.15/M output, the nearest publicly hosted 12B Gemma sibling, at its FULL catalog pair, the same explicit reference that the published google/gemma-4-12B-it rows cygnet, jev-omni, winnow-12b, torchcast-decision-12b and the live decisio v0.8.0 and v0.9.0 rows already carry. It is applied explicitly, as a labelled estimate; no base key is registered for it. Estimated, not charged, not a quote from the author or any host. Tokens as reported natively by the usage block of the author's server: 715,443 input and 1,485 output tokens across the 1,485 answered items (1 reported output token per answered row for the option-label readout; no generated prose). The native input unit counts header, state and question separately without the system turn or chat template. It is not a count of every token processed internally: the server can also register the state prefix through a warm-up, and that work is not added to native usage. Native input usage equals our first-party offline count exactly (0 mismatches over the answered items; our sealed-input count before the run was also 715,443). The 15 items refused HTTP 422 carry no usage and count 0 tokens, so the common 1,485-item basis (v1.6.30) has the same totals, 715,443 input and 1,485 output tokens. The zero-output convention for native-probability rows is NOT applied: the server reports a real output count of 1 per answered row (the same treatment as the live decisio rows, seb-9b-v2 and KnowLine, which price their 1 output token per row), and the full output price is used. Measured arithmetic: USD 0.024239 per 1,000 decisions on the common 1,485-item basis = 0.375248x the unchanged USD 0.06459465517241379 cap; separately USD 0.023997 per 1,000 over all 1,500 rows = 0.371495x. WITHIN COST CAP on both bases: within the automatic cost treatment; no cap exception or top-up.

Measured p50 latency: 0.058 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)

Ranking and cap use: 0.27 s (est. incl. network (2x + 150 ms))

Self-hosted model: measured p50 0.058 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.27 s. Model authors can ask us to time a hosted endpoint instead (on request).

x2 + 0.15 s (assumption, not measured)

Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (97,887 MiB, driver 580.178.04), FIN-03 (Finland, EU), PAYG non-spot, instance type 1RTXPRO6000.30V, USD 2.558095890410959/h recorded by the runner, 150 GB disk. One VM for this row, one attempt. Measured 25 Oct 2026, VM window about 23:26-23:39Z, about USD 0.55 GPU rental (elapsed time at the recorded hourly rate). Author server at 127.0.0.1:8000, network cut before sealed input (offline proof and cut proof receipts), read-only model and input, private output, HF offline flags, no credentials. Before sealed input: model files re-hashed against SHA256SUMS; the dependency Triton gate, a pre-warm and the smoke passed (the smoke includes both 422 over-limit probes at 32,768 and 80k tokens); the TTL fit gate admitted the run. Input sha256 dfd36720077ef7c41ace367524896507c7f44ce71306fb3e659e8b85069d995b (v1.6-regular-20261012) was re-checked inside the container (match). Complete 1,500-row run, rc 0. Raw output sha256 e9c889e1a138c875eb52521748e63b4344043c7a5598527c073172dd1c81594f (the pull-hash receipt). Node-shred and local-shred receipts verified; exact-id teardown receipt confirms the VM closed. GPU rental is separate from the benchmark's estimated model tariff.

Measured latency over 1,500 decisions: p50 0.058 s · p95 0.099 s

0 decisions over 10 s.

A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.

Published source