Jeffan-31b-1710v1.7 · 10 s rule

Composite (secondary)

48.7 · Composite rank #49

Cost and measurement conditions

$0.081 per 1,000 decisions (estimated)

ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module (sha256 aa2afcd5a58d0c4cc0e3d6867d590c70b6ac58e971410c27faf9628f97bcf8b7, re-frozen 22 Oct 2026): base string google/gemma-4-31B-it priced as google/gemma-4-31b-it USD 0.09/M input, USD 0.34/M output. The exact case of 31B is required by the frozen module; it resolves to the lowercase google/gemma-4-31b-it listing at USD 0.09/M input and USD 0.34/M output, as required by the frozen pricing policy. Estimated, not charged, not a quote from the author or any host. Tokens as reported by the author engine through the shim: input_tokens includes the rendered prompt, chat template and options; output_tokens = 0 is our shim's labelled convention because the engine reports only input usage. The 31b max_tokens=1 readout is not counted as output. Reported input usage is 2,525,873 tokens: 1,969,524 first-read tokens plus 556,349 tokens from 581 conditional rereads. The offline check has offset 0 and 0 item-level mismatches, within the preregistered band 1,969,524-3,459,523. The 15 standard long items contribute 1,191,112 first-read tokens; no reread of those items fits. Excluding them as for every row on the common 1,485-item cost basis (v1.6.30) leaves 1,334,761 reported input tokens, INCLUDING the 556,349 reread tokens, and 0 output tokens. The preregistered all-rereads common-basis bound (2,268,411 tokens, USD 0.137479/1,000) is not the measured usage. Measured arithmetic: USD 0.080894606 per 1,000 decisions on the common basis = 1.252342099x the unchanged USD 0.06459465517241379 cap; separately USD 0.151552380/1,000 over all 1,500 items = 2.346206193x. OVER COST CAP: ranked_over_cap, shown only in uncapped views and excluded from official cost-capped views; no cap exception or top-up (ruling on file). COST ON THE COMMON BASIS: USD 0.080895 per 1,000 decisions on 1,485 items, 1.252 times the official cost cap USD 0.064595; estimated, not charged. OVER THE OFFICIAL COST CAP, DISCLOSED: ranked_over_cap; outside the official-caps views.

x2 + 0.15 s (assumption, not measured)

Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (97,887 MiB), FIN-03 (Finland, EU), PAYG non-spot, instance type 1RTXPRO6000.30V, USD 2.5741917808219177/h recorded by the runner. One VM for both rows back to back, 31b then 4b. Engine and loopback shim in a --network none --read-only container per row; serial requests, read-only code/model/input, blank credential variables, HF offline flags. In-container network cut proven before sealed input: only loopback, no default route, DNS and TCP fail; code inventory/hashes and model files checked. Attempt 2 measured 24 Oct 2026 07:59-08:36Z; this row's measurement container 1035 s, run cap 1945 s, complete=true, deadline_hit=false, rc 0; scored loop 862.812179 s over 1,500 requests. Frozen v1.6-regular-20261012 self-hosted input sha256 dfd36720077ef7c41ace367524896507c7f44ce71306fb3e659e8b85069d995b, checked in the container. Raw output sha256 5f578c036684557641cbfa55bfa043ed209775cbf793d05c4f78c1f4ce7c4fe0. Row cleanup verified; final exact-id VM teardown confirms provider absent and guard closed at 08:36:06Z. GPU rental is not the benchmark's estimated model tariff.

Model and serving disclosure

THE AUTHORS' OWN DECISION INDEX ENGINE, run unmodified: tattrongvu/Jeffan-31b-1710 at cebfa86c2bed2b5c83b12e44de4374b20c87fd79, three author package files (__init__.py, engine.py, render.py), the three DI-kit base files at f8552a13b643d39f61185f5951bb5fed16bedf51 and our transport-only jeffan_shim.py. Exact ten-file code manifest sha256 f38f38d69571180283539bb9d8fad4412a6376c5d010bbe53919f8a81a5b8004, hash-checked in the container. dm_decision_engine.DecisionEngine(max_len=98304, max_batched_tokens=4096, reread_below=0.9, gpu_mem=0.85), vLLM 0.29.0 bf16, prefix caching, causal attention; conditional read-twice when first-read top-1 probability is below 0.9 and the doubled prompt fits. The predeclared memory ladder was not used. Python 3.12.3, torch 2.13.0+cu130, transformers 5.20.0, tokenizers 0.23.3, huggingface-hub 1.33.0. The author verified this stack on B200 only; this measures RTX PRO 6000. Base image vllm/vllm-openai:v0.29.0@sha256:c2914767605584b6d8f45686b82de173ecc99e781897aa3d0a66dacd72c51ae1; derived image sha256:b911dd6884706894279758c428e1a8884d4b995371fc4cf09c89136ee825fffd, hash-pinned wheel overlay. The submitted code was reviewed and cleared at the exact pins with the reviewed safeguards; evaluator exposure and the device were cleared for both attempts. Source review records dormant Hub-capable hf_hub_download/from_pretrained paths; the model argument is a hash-checked local directory, revision=None, offline flags set and network disabled. No author shell/subprocess, eval/exec, pickle or trust_remote_code path on the reviewed serving path. The 4b vLLM plugin was excluded and never executed. The runner and device selection change were reviewed; the amended runner was rechecked with the suite rerun and amendment recorded before retry. Weights: Apache-2.0 at this pin (earlier CC BY-NC 4.0 pins are not measured), public, ungated; bf16, 31,273,086,512 parameters, 62,546,338,264 weight bytes. FIRST MEASUREMENT of Jeffan-31b-1710 (our own intake). AUTHOR'S OWN ENGINE on our rented hardware, one request at a time; our harness replaces the author's Decision Index pipeline. NATIVE OUTPUTS: next-token probabilities over the option codes, choice argmax and probabilities, noul P(true), score expectation and probabilities; the first-party transport shim adds no prompt text or probability arithmetic. AUTHOR EXPOSURE: the preregistration records training data as undisclosed (Decision Index note) and author-reported Decision Index suite figures; no overlap or contamination audit was run by us. The author's reported figures are not independently verified by us. CONTEXT: engine max_len 98,304 prompt tokens; no lower evaluator-imposed cap and no truncation. All 1,500 items answered HTTP 200, 0 refusals; longest first read 81,715 tokens. The card states inputs up to 8,192 tokens and the author validated at most 32,362 first-read tokens; these long items exceed that validation. 10 s RULE: exactly 15 decisions over 10 s, the standard long items at positions in the pre-registered long-item set; these count wrong in the v1.7 share view (frozen v1.6 and v1.7-off views apply no time limit). Maximum 44.157234 s, median 0.122518 s, p95 0.358708 s. DETERMINISM: first 50 positions run twice, 50/50 bit-identical, 0 argmax flips, maximum probability difference 0.0. ATTEMPT HISTORY: attempt 1, pod window 24 Oct 2026 05:52-06:21Z (about USD 1.24) on a separate disposable GPU instance, stopped the 31b row after 1,057 of 1,500 items at the runner's own time cap. The smoke under-predicted long-item latency because repeated probe prompts hit the prefix cache. The partial is retained as a receipt only, never scored or merged; the 4b row had not started. Amendment 3 corrected the first-party runner to use cold nonce-bearing probes, maximum anchor latency and a long-item budget floor; a further review preceded the retry. evaluator exposure and device ruling admitted the fresh full 31b run as the final same-use retry; 4b remained a first use. Attempt 2 is this measurement, both rows back to back on new a separate disposable GPU instance, 24 Oct 07:59-08:36Z, about USD 1.57 rental. Between attempts only our first-party runner (smoke probes and fit gate, pre-registration Amendment 3) changed; the scoring harness, model, author code, input and engine settings are unchanged, and the pinned image stack was rebuilt on the new VM. OVER COST CAP: measured USD 0.080894606 per 1,000 decisions on the common basis = 1.252342099x; ranked_over_cap, uncapped views only (ruling on file).