baselabs joey-1 (2B)v1.7 · 10 s rule
Data: JevBench v1.8.1, published 2026-10-30 · method v1.7 (10 s rule), measured under the rule
Capability 57.8 · Not ranked in the current release
Measured on the 27 Oct 2026 draw, which is not yet equated to the reference scale; stronger anchor runs for that draw are being measured. Shown unranked until then.
Last measured: 2026-10-29 · measurement release: v1.8.1
MIT declared on the model card; no LICENSE file in the repository
intelligence
26.4
calibration
89.2
speed
92.1
cost
89.1
Composite (secondary)
15.6 · Not ranked on this board
Cost and measurement conditions
$0.0023 per 1,000 decisions (estimated)
GPU-time estimate from v1.8.0: one monthly-rented RTX PRO 6000 96 GB, EUR 999/month / 730 hours, ECB USD 1.1133/EUR dated 28 Oct 2026, USD 1.5235/hour; average load 70%. Architecture: small qwen, 2B active parameters; throughput 100000 tokens/s (architecture-class calibration, 29 Oct 2026, efficient-engine prefill; capped at 100,000 tokens/s). The row's own scored cost basis: 568,911 input + 1,489 output tokens over 1,489 decisions. Generated tokens are counted at the same throughput as prefill. GPU-time cost USD 0.00231593697698466/1,000; old list-price estimate USD 0.00774151779717931/1,000 (deepinfra:Qwen/Qwen3.5-2B at USD 0.02/M input and USD 0.1/M output). Applied min(old list-price estimate, GPU-time cost) = USD 0.00231593697698466/1,000. Estimated, not a bill. USD 0.00231593698 per 1,000 decisions on the frozen 1,489-item common set of this draw (USD 0.00230963809 over all 1,500 items).
p50 latency: 0.2 s
Self-hosted latency estimate under the shared method (assumption, not measured)
Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition (97,887 MiB reported, driver 580.178.04), FIN-03 (Finland, EU), instance type 1RTXPRO6000.30V, image 24.04.cuda13.0. Measured 29 Oct 2026. The launcher and engine run in a docker container with network none, read-only root, tmpfs scratch, read-only model and input, empty credential variables, HF offline flags, one request at a time, timed over loopback on the VM. Local hash-verified weights and retained runtime author-source, tokenizer and template hashes; weights-verify reports eight files verified. Public smoke passed before sealed input: timed probes at 256/2,048/8,192/16,384/32,768 tokens returned HTTP 200 with native usage equal to offline counts after separately recorded warm-ups. The slowest timed 32,768-token anchor was 1.370 s; this is a public synthetic timing, not a scored latency percentile. Public 32,769- and 82,000-token probes returned native HTTP 422, as did the unsupported 11-level score probe. Frozen-adapter public noul/choice/score/255-choice/32,768-token-state/Unicode-JSON probes passed with native usage equality. Smoke launcher info records noncausal attention, mask installed, FP32 slot_lm readout, no calibration and batched fork mode. TTL-fit admitted with 3,433.95 s remaining against 1,500 s required, forecasting 0 items over 10 s; this was an admission estimate, not a scoring exclusion. The scored server passed a separate public pre-warm. Complete 1,500-row run; scored loop 71.03320074081421 s. Raw output sha256 e275672c1013d9662329c54cafd47f9803a6af3d4396dca912c078ebee248691 (independently verified sealed artifact). Row-completion reports complete, teardown_confirmed, node_shred_verified and local_shred_verified true; node-shred and local-shred receipts verified, and teardown receipt confirms the exact owned VM closed. GPU rental is separate from the benchmark's estimated model tariff.
Measured latency over 1,500 decisions: p50 0.046 s · p95 0.053 s
0 decisions over 10 s.
A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.