KnowLine 31B-Gen1v1.7 · 10 s rule

Composite (secondary)

44.4 · Composite rank #53

Cost and measurement conditions

$0.093 per 1,000 decisions (estimated)

ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module (sha256 aa2afcd5a58d0c4cc0e3d6867d590c70b6ac58e971410c27faf9628f97bcf8b7, re-frozen 22 Oct 2026): base string google/gemma-4-31B-it (the card's base_model, in the author's capitalised spelling) priced as google/gemma-4-31b-it USD 0.09/M input, USD 0.34/M output. The exact case of 31B is required by the frozen module: it resolves to the lowercase google/gemma-4-31b-it listing, while the lowercase spelling raises 'base model lacks an explicit market-listing decision' here (the opposite of the 26B row). No explicit reference or override is used. Estimated, not charged, not a quote from the author or any host. Tokens as reported by the usage block of the author's server: input_tokens = SGLang prompt_tokens, including the chat template and the read-twice layout; output_tokens = 1 per question (the author's accounting for one label, not a generated explanation). Read twice is on for text-only requests of <= 3,000 read-once tokens: 1,485 items read twice, the 15 standard very long items read once. The native usage and the offline counter agree item by item with offset 0 and 0 mismatches: 2,722,505 input and 1,500 output tokens over all 1,500 answered items (byte-identical counts to the KnowLine-26B-A4B-Gen2.5 count with the same Gemma tokenizer). No refusals. The 15 standard long positions count for no row's cost on the common 1,485-item cost basis (v1.6.30); they carry 1,190,988 input and 15 output tokens, so common-basis input tokens = 1,531,517, with 1,485 output tokens. Measured arithmetic: USD 0.093159212 per 1,000 decisions on the common basis = 1.442212392x the unchanged USD 0.06459465517241379 cap; separately USD 0.163690300/1,000 over all 1,500 items = 2.534115239x. The preregistration predicted about USD 0.093 and about 1.44x on the 26B-class counts because the short items are read twice; the measurement matches. OVER COST CAP: ranked_over_cap, shown only in uncapped views and excluded from official cost-capped views; no cap exception or top-up (ruling on file). COST ON THE COMMON BASIS: USD 0.093159 per 1,000 decisions on 1,485 items, 1.442 times the official cost cap USD 0.064595; estimated, not charged. OVER THE OFFICIAL COST CAP, DISCLOSED: ranked_over_cap; outside the official-caps views.

x2 + 0.15 s (assumption, not measured)

Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (97,887 MiB), FIN-03 (Finland, EU), PAYG non-spot, instance type 1RTXPRO6000.30V, USD 2.5991917808219176/h recorded by the runner. One VM for this row only (attempt 2; attempt 1 used a separate VM). SGLang (127.0.0.1:9080) and author front end (127.0.0.1:8000) in ONE --network none --read-only container; driver requests serial over loopback. Read-only code, model and input, blank credential variables, HF offline flags. In-container network cut proven before sealed input: only loopback, no default route, DNS and TCP fail; code inventory/hashes and model files checked; weights re-hashed against the manifest (24 files, 0 failures). SGLang /server_info sizing gate passed before sealed input (SERVER_SIZING_OK: version 0.5.21, swa_full_tokens_ratio 0.15, mem_fraction_static 0.85, dtype auto, kv_cache_dtype auto, no quantization, FP32 LM head on, max_total_num_tokens 106,983, max_req_input_len 106,977). Attempt 2 measured 24 Oct 2026 (reservation 09:49:54Z, VM created 09:51:04Z, teardown confirmed 10:25:27Z); this row's measurement container 1,401 s, run cap 5,036 s, complete=true, deadline_hit=false, rc 0; scored loop 1,252.397 s over 1,500 requests. Frozen v1.6-regular-20261012 self-hosted input sha256 dfd36720077ef7c41ace367524896507c7f44ce71306fb3e659e8b85069d995b, checked in the container. Raw output sha256 54da5a6239bb6d60a77d5fbed9541c4ed3aded49f9e6f1961d46e1826cc9b323. Node-shred receipt verified; row cleanup verified; exact-id VM and volume teardown confirms provider absent and guard closed. GPU rental is not the benchmark's estimated model tariff.

Model and serving disclosure

THE AUTHORS' OWN SERVER, run unmodified: PelaAI/KnowLine-31B-Gen1 at 986803e4037f7778d24623184729a31202e3d302, knowline_server.py sha256 13ca50c4001fac0aa3899a35ff01e60f311132c571b17a8c3dc7aa6001ad2f3f (1,156 lines) and knowline_layout.py sha256 2e839fa24f1ad9c980898b2a9aa62e2154ba3f57618bd5f2fedb5756f61f062f (242 lines), plus the data file temperatures.json sha256 463a46c5ea253649bd5f43809655637d2d41939e503c81ddd01b28355d736f62 ({"score": 0.6944}), all mounted read-only at /model over stock SGLang 0.5.21. Only these two author Python files shipped; knowline_engine.py and serve_knowline.sh were excluded and never executed (the SGLang flags follow the author's serve_knowline.sh, plus our one pooling flag --swa-full-tokens-ratio 0.15, see below). Commands: env HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 PYTHONDONTWRITEBYTECODE=1 python3 -m sglang.launch_server --model-path /model --served-model-name m --tp 1 --mem-fraction-static 0.85 --enable-fp32-lm-head --swa-full-tokens-ratio 0.15 --host 127.0.0.1 --port 9080 ; env HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 PYTHONDONTWRITEBYTECODE=1 python3 /model/knowline_server.py --model /model --backend sglang --url http://127.0.0.1:9080 --served-model-name knowline-31b-gen1 --temperature 1 --workers 16 --read-twice --twice-max-tokens 3000 --host 127.0.0.1 --port 8000 --temperatures /model/temperatures.json --box-prompt gemma --box-coords yxyx. Never passed: --allow-url-assets, --referenced-fields, --header, --no-read-twice, --debug-refusals, --box-calibration, --box-prompt-template (asserted on the rendered launch script; /health after the measurement shows referenced_fields false, url_assets false, no box calibration). Image: lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469, derived image sha256:579dd709d9118e745c9be62fc134fc6aac272a2ee75834da15f7cd235a025a4e (only blank-credential ENV, no package installs). Python 3.12.3, sglang 0.5.21, torch 2.13.0+cu130, transformers 5.12.1, tokenizers 0.22.2, flashinfer-python 0.6.18, sglang-kernel 0.4.7. Code gate: cleared by the benchmark owner on the disclosed patterns. DISCLOSED PATTERNS: a new dormant runtime download path, requests.get of http(s) image URLs (knowline_server.py _fetch, line 383), reachable only behind the command-line flag --allow-url-assets (off by default, never passed, container without network); dormant Hub-capable from_pretrained loaders in the unused --backend hf class; served tokenizer load from local /model under offline flags; HTTP client exclusively to loopback SGLang /generate (scoring and the dormant box path); local listening socket on 127.0.0.1. About 800 added lines (image, box and referenced-fields code) are not reachable from text-only requests; the one text-path change (noul criteria values rendered through render_value) leaves string, null and number criteria byte-identical and is mirrored by our offline token counter. Delta source review PASS WITH CONDITIONS (flag discipline, hash-checked snapshot, in-container offline proof, all enforced by the runner), the runner was reviewed and findings fixed. OUR ONE LAUNCH-FLAG CHANGE (pre-registration Amendment 3): attempt 1 stopped in the synthetic smoke because the default SGLang pool sizing (--swa-full-tokens-ratio 0.8) gave only 29,717 full-attention cache tokens; attempt 2 adds --swa-full-tokens-ratio 0.15, which only reallocates the same KV memory between the full-attention and sliding-window pools (live capacity max_total_num_tokens 106,983, max_req_input_len 106,977); no change to weights, BF16 precision, KV dtype, mem-fraction, context configuration or the 1,024-token attention window; bit identity against the default sizing is not promised. The amendment was reviewed and the final hashes checked (final runner sha256 18c051f1b72ec682339a11eb90025eb4c73fc4b526932c24f3faecf557fc270d), covered by the standing clearance for this amendment. Weights: Apache-2.0, public, ungated, 62,546,336,352 weight bytes in 13 BF16 shards, hash-checked file by file on the node; prereg manifest sha256 0444b046991995634646ef018d16103292eb9807147d0287898b2380b95ed98d; the shards share no LFS sha256 with the weights of the measured KnowLine 4B, 26B and 35B rows. FIRST MEASUREMENT of KnowLine-31B-Gen1 (our own intake). FIRST DENSE-GEMMA KnowLine: new weights (Gemma 4 31B-it fine-tune, LoRA rank 32 merged, then averaged 0.75 fine-tuned + 0.25 base per tensor, readout temperature 1.44 folded into the final norm and soft-cap), not a new serving version of an earlier row; the 4B, 26B and 35B KnowLine rows are different weights. AUTHOR'S OWN SERVER on our rented hardware, bf16 (no FP8; the author documents bf16 only). The server and layout files are not the bytes measured for the earlier KnowLine rows (1,156 lines against 367; see code_revision); on our text items the prompts follow the same read-twice layout. NATIVE OUTPUTS: label-token logprobs, label-only softmax at temperature 1 for choice and noul questions and at the author's 0.6944 for score questions (temperatures.json, effective 1.0 on top of the folded 1.44; both are ranking-relevant behaviours of the author's server); choice argmax, Noul P(true), score expectation; header off, no rotation, no calibration file. AUTHOR EXPOSURE (card, self-reported, not verified): Decision Index 0.3 public suite 70.13, self-run read twice; the card also reports JevBench / JevBench-hard evaluations; the training data is that of KnowLine-26B-A4B-Gen1 (906,122 rows, about 33% targeting Decision Index benchmarks in their request format, including train splits of public datasets rewritten in that format and synthetic items in the same format); the authors checked it against the Decision Index 0.3 rows, their own evaluation sets and public benchmarks including JevBench and removed 491 near-duplicate rows; the 0.75 averaging weight was chosen on the authors' own evaluation sets; LLM teachers labelled the synthetic tasks and not all labels were checked by a human; the card says part of the public-suite score comes from benchmarks whose tasks are in the training data. The authors say no Jev or other decision-model outputs were used as training labels. No overlap or contamination audit was run by us. The author's H20 timing (340 ms median, 3,977 ms at the 95th percentile) does not transfer to this RTX PRO 6000 measurement. CONTEXT: SGLang configuration limit 262,144 tokens, with live KV capacity max_total_num_tokens 106,983 and max_req_input_len 106,977 at our pooling flag; no author-imposed lower cap, no truncation and 0 refusals; the longest item was 81,706 tokens. 10 s RULE: exactly 15 decisions over 10 s, the standard very long items at positions in the pre-registered long-item set, cold prefills of 76,531-81,706 tokens, up to 59.753 s each; these count wrong in the v1.7 share view (the frozen v1.6 and v1.7-off views apply no time limit). No decision came near the adapter's 120 s timeout. Maximum 59.753078 s, median 0.255530 s, p95 0.469112 s. DETERMINISM: first 50 positions run twice, 40/50 bit-identical, 0 argmax flips, maximum probability difference 0.0141815; the scored raw is the first pass, so scores are unaffected; bit reproducibility is not established and the cause was not investigated. ATTEMPT HISTORY: attempt 1, pod window 24 Oct 2026 08:55-09:07Z (about USD 0.49 on the runner ledger) on a separate disposable GPU instance, stopped in the synthetic smoke before any held-out item: at the default pool sizing SGLang reported 29,717 full-attention cache tokens and refused the 32,000-token probe (HTTP 422, maximum allowed input length 29,711). No sealed byte was sent and exposure stayed unspent; the attempt is retained as a receipt only, never scored or merged. pre-registration Amendment 3 changed only our SGLang launch flag (--swa-full-tokens-ratio 0.15) and added a server_info sizing gate (max_total_num_tokens >= 90,000, max_req_input_len >= 85,000) that precedes the first sealed byte; a further review and a final-hash check preceded the retry. evaluator exposure and device ruling covered attempt 1 and ruling on file admitted attempt 2 as the final attempt; evaluator exposure and device ruling was unspent at attempt 1 and is consumed by attempt 2. Attempt 2 is this measurement on new a separate disposable GPU instance, 24 Oct 09:49-10:25Z, about USD 1.54 rental. Between attempts only the pooling flag and our first-party runner and tests changed; the scoring harness, model, author code, input and all other engine settings are unchanged, and the pinned image stack was rebuilt on the new VM. OVER COST CAP: measured USD 0.093159212 per 1,000 decisions on the common basis = 1.442212392x; ranked_over_cap, uncapped views only (ruling on file).