Seb-9B-v2 (ironbcc, Qwen3.5-9B fine-tune, one forward pass, stock vLLM, self-hosted)v1.7 · 10 s rule

Data: JevBench v1.7.0, published 2026-10-24 · method v1.7 (10 s rule), measured under the rule

Capability 64.8 · rank #45 of 157 on the open-weights board

Last measured: 2026-10-23 · measurement release: v1.7.0

Apache-2.0 (LICENSE in the repo and HF metadata; base Qwen/Qwen3.5-9B Apache-2.0), public, not gated

Base model: undisclosed

Predecessor: Seb-9B. Both measurements remain separate rows.

intelligence

46.1

calibration

83.5

speed

92.0

cost

48.9

Composite (secondary)

50.0 · Composite rank #46

Cost and measurement conditions

$0.051 per 1,000 decisions (estimated)

ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module (sha256 aa2afcd5a58d0c4cc0e3d6867d590c70b6ac58e971410c27faf9628f97bcf8b7, re-frozen 22 Oct 2026): base Qwen/Qwen3.5-9B (declared by the author on the card and in merge_record.json) -> qwen/qwen3.5-9b USD 0.10/M input, USD 0.15/M output. The same estimate as the v1 row seb-9b. Estimated, not charged, not a quote from the author or any host. Tokens as reported by vLLM's own usage block: 748,494 input tokens on the 1,485 answered items (equal to our offline per-item count item by item, 0 mismatches, pre-registered; the same total as the v1 row, as the tokenizer and template are unchanged), and 1 output token per answered row (1,485: one forward pass that reads the label logprobs); the 15 refused items count 0 tokens. COST ON THE COMMON BASIS: USD 0.050554 per 1,000 decisions on 1,485 items, 0.783 times the official cost cap USD 0.064595; estimated, not charged.

Measured p50 latency: 0.035 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)

Ranking and cap use: 0.22 s (est. incl. network (2x + 150 ms))

Self-hosted model: measured p50 0.035 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.22 s. Model authors can ask us to time a hosted endpoint instead (on request).

x2 + 0.15 s (assumption, not measured)

Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (97,887 MiB), FIN-03 (Finland, EU), pay-as-you-go non-spot, instance type 1RTXPRO6000.30V at USD 2.574/h. Stock vLLM 0.30.0 (127.0.0.1:8000, digest-pinned image) and our shim (the /v1/systemone endpoint) run in containers with network none, driven one request at a time by the unchanged pinned pod_driver_v16. The standard self-hosted x2 + 0.15 s latency adjustment applies. Network cut proven from inside the container before any item hash or model load (loopback only, no default route, DNS and TCP fail); read-only code, read-only model and input, empty credential variables (names-only proof), HF offline flags. Measured 23 Oct 2026 (VM 07:12-07:25Z; measurement wrapper 250 s; scored loop 63.3 s per the receipt) on the frozen v1.6-regular-20261012 self-hosted input (sha256 dfd36720077ef7c41ace367524896507c7f44ce71306fb3e659e8b85069d995b, re-checked inside the container). The input was shredded from the node and its absence verified, and the VM was removed by exact id afterwards (teardown confirmed 07:25Z).

Measured latency over 1,500 decisions: p50 0.035 s · p95 0.069 s

0 decisions over 10 s.

A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.

Model and serving disclosure

No author code runs at all (static review PASS: the repo's four serve/*.py files are excluded from the manifest and never copied to the node; the run fails if any .py/.sh/.so/.pyc/.pkl/.pt/.bin exists in /model). Weights: ironbcc/Seb-9B-v2 at ca903824a11dd763a8e0dec5bce5e13e3e4166db (Apache-2.0, public, not gated; Qwen/Qwen3.5-9B plus a merged LoRA, scale 20, 248 modules; 14 manifest files, 4 bf16 shards = 19,306,311,000 bytes; manifest sha256 d138599972ed6caa640987b3bd175a24e8f0d76e4485317d1de6240044d660d4; every file sha256-verified against the Hub oids). The Hub HEAD later moved to c01ef79eb54b491c83ed1260ab8444cbc662e65c by README, blog and media commits only: the 19 weight, config and tokenizer oids are identical at both commits, and the run loads exactly the pinned commit. config.json, chat_template.jinja, tokenizer.json, tokenizer_config.json and the preprocessor files carry the same git oids as the v1 row seb-9b; vocab.json and merges.txt on the Hub are truncated at 1,048,576 bytes (unused by the fast tokenizer; equivalence with v1 proven offline on 467 texts and by the usage match below). Served by the stock vLLM 0.30.0 OpenAI server (image pinned by digest vllm/vllm-openai:v0.30.0@sha256:8a69ffad015f138d7170c4ddc429e230a3bc1c1719f67e14324749df200a4b90, no derived image) with the author's published command (vllm serve /model --served-model-name seb-9b-v2 --logprobs-mode processed_logprobs --max-logprobs 32 --generation-config vllm --max-model-len 32768 --dtype bfloat16 --max-num-seqs 128 --enable-prefix-caching --gpu-memory-utilization 0.85 --host 127.0.0.1 --port 8000); the only delta from the v2 card is --gpu-memory-utilization 0.85, kept from the v1 recipe (it only sizes the KV cache). vllm 0.30.0, torch 2.13.0+cu130, transformers 5.17.0, tokenizers 0.23.2, flashinfer 0.6.18.post1. The /v1/systemone wire layer is OUR reviewed shim seb_systemone_r56.py (sha256 e617f911e9218359b35eb8d7761ae6674c75631398efdd76996b1741e6a17b99, byte-identical to the v1 row's): it builds the author's documented chat request (max_tokens 1, temperature 0, top_logprobs, structured_outputs.choice, thinking off) and renormalises exp(top_logprobs) over the option labels; it applies no temperature and no rule of its own. The runner was reviewed and findings resolved in the preregistration amendments before the run. FIRST MEASUREMENT OF THIS VERSION. NEW VERSION OF AN EXISTING MODEL: Seb-9B-v2 has new weights (all four shards differ from ironbcc/seb-9b); the v1 row seb-9b measured 20 Oct 2026 stays as it is. The submission reference is not recorded in the files read for this entry. Whether this row supersedes the v1 row or stands beside it is a release-owner decision. SECOND USE OF THE DRAW FOR THE FAMILY: evaluator exposure and device ruling admitted this as a same-author second use (v1 consumed the frozen draw on 20 Oct, on file; the v2 weights were published after that score), with disclosure; any further ironbcc row goes to the next draw. AUTHOR CONTEXT CAP: at the card's 32,768-token limit vLLM refuses the 15 items of this draw that are 76,344-80,504 tokens long (HTTP 400, mapped to a per-question refusal HTTP 422 by our shim) at positions 65, 169, 474, 482, 515, 573, 580, 729, 833, 899, 951, 1057, 1116, 1198, 1418, identical to the v1 set; they count as wrong. The card also says Seb-9B-v2 was trained and evaluated with prompts of up to 1,024 tokens and that --max-model-len only sets the server limit; answered items run up to 1,875 tokens (mean 504, median 455), so some answered items are longer than the author's stated training and evaluation length. We used the card's 32,768 setting, as for v1. OWN HTTP SHIM: the wire layer is ours (see code_revision). TRAINING DATA AND SELF-REPORTED NUMBERS (the author's own card, unverified): the card reports Decision Index 0.3 public 59.16 (against 57.96 for hosted Jev 1.13 and 60.70 for Jev 1.14); this is not a JevBench result. We did not audit training data or overlap with the JevBench public 231 and ran no overlap check; no JevBench-use statement for v2 was recorded in the files read. HARDWARE: unlike v1 (A100 80 GB), this row ran on one RTX PRO 6000 (sm_120); latency reflects that card and the Gated-DeltaNet kernels ran clean (usage matched our offline count on all 1,485 answered items). 10 s RULE: max measured decision latency 0.138 s, 0 decisions over 10 s. DETERMINISM: first 50 positions run twice, 50/50 bit-identical, 0 argmax flips, max probability difference 0. No outside request for this model. PREDECESSOR: seb-9b (/jev-models/seb-9b); both measurements remain separate rows.

Published source