basal-omni-1.0-ultra (Remek Kinas, Gemma-4-31B-it + merged LoRA, typed-decision readout, self-hosted)
Data: JevBench v1.6.39, published 2026-10-22
Capability 82.5 · rank #3 of 139 on the open-weights board
Last measured: 2026-10-17 · measurement release: v1.6.28
Apache-2.0 (model card and HF metadata; base google/gemma-4-31B-it Apache-2.0), public, not gated
intelligence
74.1
calibration
91.0
speed
87.3
cost
50.4
Composite (secondary)
71.7 · Composite rank #5
Cost and measurement conditions
$0.045 per 1,000 decisions (estimated)
ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module: declared base google/gemma-4-31B-it -> google/gemma-4-31b-it USD 0.09/M input, USD 0.33999999999999997/M output (frozen at the 25 Sep cut-off, I-3). Estimated, not charged, not a quote from the author or any host. USD per 1,000 decisions over the common 1,485-item cost basis. Tokens as reported by the author's own server (usage.input_tokens = tokens of the rendered chat prompt incl. system prompt, options and the answer prefill; output_tokens 0, nothing is generated - the model reads the option-code logits at one answer position in one forward pass). The offline count through the author's own renderer and the pinned tokenizer predicted the server's numbers exactly, item by item: 744,183 input tokens over the 1,485 answered items. COMMON COST BASIS (v1.6.30): cost per 1,000 decisions is computed on the common 1,485 items of this draw, leaving out the 15 very long items (at least 76,358 input tokens as counted by the rows that answered them). They are the 15 items the Jev reference refused with max_tokens_exceeded, which the other rows that did not answer them refused or failed on; their tokens count for no row, whether the row refused or answered them, and a refused item still counts as a wrong answer for Intelligence. This row: USD 0.045102 per 1,000 decisions (0.698x the official cost cap USD 0.064595), against USD 0.044651 over all 1,500 items (v1.6.29).
Measured p50 latency: 0.102 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.35 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.102 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.35 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
Evaluator-owned rented Lium GPU pod, 1x NVIDIA H200 141 GB (sm_90, driver 580.173.02), Finland (EU), SPOT tier, USD 3.99/h, 17 Oct 2026; sealed run 08:59:30-09:25:01Z, one request at a time, no resumes. Speed is as measured on that card; the card's own figure is 80 ms p50 on an H200 (measured here: raw p50 0.107 s / p95 0.216 s over all 1,485 answered items; the Speed axis uses the official latency subset given in the row note, with the standing x2 + 0.15 s self-hosted adjustment on top). DEVICE DEVIATION: the pre-registered device was one non-preemptible 80 GB H100; no single-GPU H100 was rentable in the EU, so the run used this one H200 on the spot tier under a pre-registration amendment written before the sealed run (the card says about 32,000 tokens fit on a 141 GB H200 and about 10,000 on an 80 GB H100; the same 15 items were predicted to be refused on either card). Speed is therefore not strictly comparable to the H100 / RTX 5090 rows. Isolation: the author's server ran inside a network namespace with loopback only, with the cut PROVEN before any sealed item was dispatched (no interfaces but lo, no routes, outbound by name and by IP both failing) and proven again immediately before the first sealed request; credential-free environment (env -i, checked), HF offline flags, weights from a locally verified offline snapshot of the pinned revision (13 files, 62,578,671,091 bytes, every sha256 equal to the author's SHA256SUMS), sealed input held in tmpfs and deleted on the pod at 09:24:55Z, server process group killed by exact pid. Transfer: HF_HUB_DISABLE_XET=1 (plain HTTPS, same verification).