KnowLine 35B-A3B-Gen1.5v1.7 · 10 s rule

Composite (secondary)

12.4 · Composite rank #144

Cost and measurement conditions

$0.323 per 1,000 decisions (estimated)

ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module (sha256 aa2afcd5a58d0c4cc0e3d6867d590c70b6ac58e971410c27faf9628f97bcf8b7): the installed module has no exact mapping for Qwen/Qwen3.5-35B-A3B; its BASE_REFERENCES['Qwen/Qwen3.5-35B-A3B-Base'] resolves to qwen/qwen3.5-35b-a3b USD 0.3125/M input, USD 1.25/M output, applied explicitly. The exact base has a cheaper DeepInfra listing which this frozen module does not map; no pricing-module change is included in this release. The frozen reference remains USD 0.3125/M input and USD 1.25/M output, 5.00 times the official cap. Estimated, not charged, not a quote from the author or any host. Tokens as reported by the usage block of the author's server: input_tokens = SGLang prompt_tokens, including the chat template and the read-twice layout; output_tokens = 1 per question (the author's accounting for one label, not a generated explanation). Read twice is on for text-only requests of <= 3,000 read-once tokens: 1,485 items read twice, the 15 standard very long items read once. The native usage and the offline counter agree item by item with offset 0 and 0 mismatches: 2,708,769 input and 1,500 output tokens over all 1,500 answered items. No refusals. The 15 standard long positions count for no row's cost on the common 1,485-item cost basis (v1.6.30); common-basis input tokens = 1,528,836, with 1,485 output tokens. The pre-registration predicts about USD 0.323 per 1,000 decisions, about 5.0 times the cost cap, because the short items are read twice and the base reference is priced higher than the 4B and 26B references; measured figures are reported separately in this row.

x2 + 0.15 s (assumption, not measured)

Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (97,887 MiB), FIN-03 (Finland, EU), PAYG non-spot, instance type 1RTXPRO6000.30V. One VM for this row only (evaluator exposure and device ruling; --rows 35b-gen15). SGLang (127.0.0.1:9080) and author front end (127.0.0.1:8000) in ONE --network none --read-only container; driver requests serial over loopback. Read-only code, model and input, empty credential variables, HF offline flags; cut proof inside the container before sealed input (only loopback, no default route, DNS/TCP fail). Measured 23 Oct 2026 (reservation 08:42Z, VM created 08:43Z, teardown confirmed 08:58:23Z); measurement container 244 s (248 s including pulls), scored loop 122.727 s; run cap 2,219 s = 1.5 x the smoke-predicted 279 s plus 120 s per long item (15) under the corrected runner. Frozen v1.6-regular-20261012 self-hosted input sha256 dfd36720077ef7c41ace367524896507c7f44ce71306fb3e659e8b85069d995b, checked in container. Node-shred receipt verified; exact-id VM teardown confirms provider absent and guard closed. GPU rental is not the benchmark's estimated model tariff.

Model and serving disclosure

THE AUTHORS' OWN SERVER, run unmodified: PelaAI/KnowLine-35B-A3B-Gen1.5 at 9adcf717361b2f7d2946d093f1e85efdf1e59010, knowline_server.py sha256 ad922e7d2d717a6b9b936983ff5e5d69534edc593ed54b58732dc8ff60baa67e and knowline_layout.py sha256 aa0fb35619f4883edd6c3bd1cc6dc259f41f082cfe0471e586c60d783da2af9d, both mounted read-only at /model over stock SGLang 0.5.21. Only these two author Python files shipped; knowline_engine.py and serve_knowline.sh were excluded and never executed. Commands: env HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 PYTHONDONTWRITEBYTECODE=1 python3 -m sglang.launch_server --model-path /model --served-model-name m --tp 1 --quantization fp8 --mem-fraction-static 0.72 --mamba-radix-cache-strategy extra_buffer --enable-fp32-lm-head --host 127.0.0.1 --port 9080 ; env HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 PYTHONDONTWRITEBYTECODE=1 python3 /model/knowline_server.py --model /model --backend sglang --url http://127.0.0.1:9080 --served-model-name knowline-35b-gen15 --temperature 1 --workers 16 --read-twice --twice-max-tokens 3000 --host 127.0.0.1 --port 8000. Image: lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469, derived image sha256:a829b8992c579540f1231a66ddd7732a470a22379d85a9f2138b6eb58bc1c187 (only blank-credential ENV, no package installs). Python 3.12.3, sglang 0.5.21, torch 2.13.0+cu130, transformers 5.12.1, tokenizers 0.22.2, flashinfer-python 0.6.18, sglang-kernel 0.4.7. Code gate: cleared by the benchmark owner on the disclosed patterns. DISCLOSED PATTERNS: dormant Hub-capable from_pretrained loaders in the unused --backend hf class; served tokenizer load from local /model under offline flags; HTTP client exclusively to loopback SGLang /generate. The source and runner were reviewed, with findings resolved in the amendments; the runner cap fix of Amendment 1c was independently reviewed (PASS) before this pod. Weights: Apache-2.0, public, ungated, 71,903,879,336 weight bytes, hash-checked file by file; pre-registration manifest sha256 61c753a9be64d055d16d69f2efa10b4346331844a447ed9799cd41357e38adab; byte-identical to Gen1 weights (revision e800968d). FIRST MEASUREMENT of this .5 serving version (our own intake). SAME WEIGHTS AS Gen1 (no earlier-generation KnowLine row exists on the board): the shards, tokenizer, configuration and chat template are byte-identical to the predecessor; the new author layout reads short text inputs twice, so this measures a new serving version, not new weights. AUTHOR'S OWN SERVER on our rented hardware, online FP8. NATIVE OUTPUTS: label-token logprobs, label-only softmax at temperature 1; choice argmax, Noul P(true), score expectation; header off, no rotation or calibration file. AUTHOR EXPOSURE (card, self-reported, not verified): the authors adopted read twice after checking their own evaluation sets and a fresh full run of the Decision Index 0.3 public suite; the card also reports JevBench / JevBench-hard evaluations. Training-data, Decision Index format-data and decontamination disclosures are inherited from Gen1; no overlap or contamination audit was run by us. The authors say no Jev or other decision-model outputs were used as training labels. The author's H20 timing does not transfer to this RTX PRO 6000 measurement. 10 s RULE: 0 decisions over 10 s, so no decision is scored wrong under the 10 s rule in the v1.7 share view (the frozen v1.6 and v1.7-off views apply no time limit); maximum 3.397 s (median 0.0476 s, p95 0.0621 s); the slowest decisions are the standard very long items of about 80,000 tokens. DETERMINISM: first 50 positions run twice, 2/50 bit-identical, 1 argmax flip, maximum probability difference 0.0395585 (the pre-registration expected 0 flips: a prediction miss; the scored raw is the first pass, so scores are unaffected; the 4B and 26B rows had 0 flips); bit reproducibility is not established and the cause was not investigated. CONTEXT: SGLang configuration limit 262,144 tokens; no author-imposed lower cap, no truncation and 0 refusals; the longest item was 80,522 tokens. SEPARATE POD: this row was measured alone on its own pod after the first pod (4B and 26B rows) ended with the 26B row flagged partial by the runner (its data are complete); the 35B exposure use was unspent at that point, the runner cap was corrected (Amendment 1c) and independently reviewed, and the run completed with complete=true, no deadline hit and rc 0 (run cap 2,219 s, scored loop 122.7 s). OVER COST CAP: read-twice with this base reference is predicted about 5.0 times the cap on the common basis; measured USD 0.322975 per 1,000 decisions on the common basis = 5.00x the cap (all rows 0.565577 = 8.76x), ranked_over_cap.