vLLM-SR d3-turbo
Data: JevBench v1.6.39, published 2026-10-22
Capability 82.6 · Not ranked in the current release
cost 6.8× Jev (v1.5 reference)
Last measured: 2026-10-19 · measurement release: v1.6.31
Apache-2.0 (weights, model card and NOTICE; base Qwen/Qwen3.8-27B family Apache-2.0). Training data not disclosed
intelligence
74.8
calibration
90.4
speed
86.0
cost
29.8
Composite (secondary)
20.5 · Composite rank #94
Cost and measurement conditions
$0.218 per 1,000 decisions (estimated)
ESTIMATE (base-model reference; self-hosted open weights, no public tariff for these weights): declared base Qwen/Qwen3.8-27B -> qwen/qwen3.8-27b USD 0.42/M input, USD 3.0/M output - the same pair the published vllm-sr-d3 row on this draw carries - x this row's own measured tokens on the common 1,485-item basis (771,190 input tokens on the 1,485 answered items; the 15 items over the author's 32,768-token default cap are refused by the author's own server with 0 tokens counted; output 0 because nothing is generated). Estimated, not charged; not a quote from the author or any host. OVER THE OFFICIAL COST CAP: USD 0.21811434343434347 per 1,000 decisions on the 1,485 common items, 3.38x the official cap USD 0.06459465517241379; ranked and disclosed following the published vllm-sr-d3 precedent. The frozen Qwen3.8-27B pair is also carried by published startlux-decision-v1.1-27b-instant and startlux-decision-v1.1-27b-autothink. ALL-ITEMS DIAGNOSTIC: USD 0.2159332 per 1,000 decisions over the full 1,500-item denominator, 3.34x the official cap; the published cost above uses the same remaining 1,485 items as every other row. All items still count for Intelligence.
Measured p50 latency: 0.144 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.44 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.144 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.44 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
evaluator-owned rented Verda GPU VM, 1x NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (FIN, pay-as-you-go non-spot), the same GPU class the published vllm-sr-d3 row was timed on (that row ran in confidential-computing mode, this one did not; raw p50 0.144 s vs d3's 0.109 s). The author's own unmodified stdlib d3_server.py (plain CUDA / PyTorch path; the author's ROCm fast path does not apply on CUDA) in an offline container with --network none (cut proven from inside the measuring container before the sealed input: loopback only, no default route, TCP and DNS fail), Hub offline mode and all credentials unset, every weight shard, readout, tokenizer, config and code file sha256-verified against the author's MODEL_MANIFEST.json; driven by the unchanged pinned pod_driver_v16. The standard self-hosted x2 + 0.15 s adjustment applies. Measured 19 Oct 2026 on the frozen v1.6-regular-20261012 self-hosted input (sha256 dfd36720...); determinism 50/50 bit-identical; VM and volume destroyed afterwards.