Arcbright Elf 1 31Bv1.7 · 10 s rule
Data: JevBench v1.7.0, published 2026-10-24 · method v1.7 (10 s rule), measured under the rule
Capability 81.8 · rank #3 of 157 on the open-weights board
Last measured: 2026-10-22 · measurement release: v1.7.0
Apache-2.0 (model card, root LICENSE and serve/LICENSE; base google/gemma-4-31B-it under the Gemma 4 licence), public, not gated
intelligence
71.9
calibration
91.6
speed
89.9
cost
49.7
Composite (secondary)
70.5 · Composite rank #9
Cost and measurement conditions
$0.047 per 1,000 decisions (estimated)
ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module (sha256 a8a05890ddab2d39d1a1bd543f2b5c83842bdc70aa54766788bc89b7a8a448d7): base string google/gemma-4-31B-it (the card's base_model) priced as google/gemma-4-31b-it USD 0.09/M input, USD 0.34/M output. Estimated, not charged, not a quote from the author or any host. Tokens as reported by the author's own server usage block (input_tokens = tokens of the rendered chat prompt incl. system prompt, options and the answer prefill; output_tokens = 1, one vLLM forward pass per answered single-question row, nothing generated). The offline count through the author's render.py and the pinned tokenizer predicted the server's numbers EXACTLY, item by item: 777,809 input tokens over the 1,485 answered items, and 15 HTTP 422 refusals at the pre-registered ids and positions.
Measured p50 latency: 0.062 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.27 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.062 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.27 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (driver 580.178.04), FIN-03 (Finland, EU), pay-as-you-go non-spot, instance type 1RTXPRO6000.30V at USD 2.525/h. The author's own `python3 -m elf_serve` from the pinned commit on 127.0.0.1:8000 inside a network-none Docker container with the cut proven from inside (loopback only, no default route, DNS and TCP fail), offline and credential-free, one request at a time. Timed over loopback on the VM. Measured 22 Oct 2026.
Measured latency over 1,500 decisions: p50 0.062 s · p95 0.114 s
0 decisions over 10 s.
A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.
Model and serving disclosure
The author's own `elf_serve` package at the pinned revision 8c7799280c8c04a3802af1feeb3a1ad40d6c4eeb, unmodified: serve/elf_serve/{__init__,__main__,joint,render,server,settings,worker}.py, byte-identical (sha256, 7/7) to the arcbright/elf-1-12b serve files measured 20 Oct; the author's serve/SHA256SUMS verifies 11/11 inside the container. Started as `python3 -m elf_serve` with ELF_MODEL pointed at the commit-pinned local snapshot and the repo-root decision_config.json (the configuration of the author's Decision Index run: hybrid-v1, online FP8 from the bf16 checkpoint, max_model_len 32,768, temperatures scalar 2.186 / choice 1.7509 / noul 1.1758 / score 2.1257, noul_commit [0.199, 0.801]); ELF_CONFIG was never set, so serve/configs/elf-1-31b-prod.json was not used. Stock digest-pinned vllm/vllm-openai:v0.30.0 image plus the same hash-pinned wheels as the 12B row (offline --no-deps install). The two audited patterns (two read-only constant nvidia-smi queries in settings.py; a snapshot_download fallback unreachable as measured, network-none container) ride along unchanged: The benchmark owner cleared these exact bytes for this one measurement before the run. Weights: the commit-pinned snapshot, every file sha256-verified on the host and again in the container. FIRST MEASUREMENT (found by our own intake: Hugging Face decision-model tag / Decision Index). AUTHOR-DEFAULT CONTEXT CAP: the shipped decision_config.json caps a request row at 32,768 tokens; the author's server returns a real HTTP 422 above it, never a truncation. The 15 items of 76k-82k tokens were refused exactly as pre-registered and are scored wrong; the largest answered item is 1,828 tokens. ONLINE FP8: the author's Decision Index configuration quantises the bf16 checkpoint to FP8 at load time; this row is that configuration. NOUL COMMIT: the shipped config moves an interior noul probability to the nearer of 0.199 / 0.801 (author default, kept). SAME CODE AS ELF 1 12B: the seven serve files are byte-identical to the measured arcbright-elf-1-12b row. AUDITED CODE PATTERNS: see code_revision. TRAINING DATA, the author's own statement: one LoRA on the attention layers merged into bf16, 38,000 rows from public datasets incl. ZefanCai/Open-Jev and Open-Jev-v1.1 plus own generators, including the train splits of six Decision Index benchmarks (HoVer, WinoGrande, When2Call, GSM8K, BANKING77, CLINC150); the author states it screened for overlap with Decision Index test items - not independently verified here. SELF-REPORTED, unverified: 84 ms median latency (FP8, one RTX PRO 6000) on the author's card. 10 s RULE: max measured decision latency 0.24 s; no decision near the limit.