Piccolo 9B (agentscope-ai)v1.7 · 10 s rule
Data: JevBench v1.8.2, published 2026-10-30 · method v1.7 (10 s rule), measured under the rule
Capability 76.6 · Not ranked in the current release
Measured on the 27 Oct 2026 draw, which is not yet equated to the reference scale; stronger anchor runs for that draw are being measured. Shown unranked until then.
Last measured: 2026-10-30 · measurement release: v1.8.2
Apache-2.0 (weights, HF LICENSE + card); FlexDM-derived layout MIT notice retained
intelligence
63.8
calibration
89.5
speed
87.5
cost
63.9
Composite (secondary)
74.2 · Not ranked on this board
Cost and measurement conditions
$0.016 per 1,000 decisions (estimated)
GPU-time estimate from v1.8.0: one monthly-rented RTX PRO 6000 96 GB, EUR 999/month / 730 hours, ECB USD 1.1133/EUR dated 28 Oct 2026, USD 1.5235/hour; average load 70%. Architecture: small qwen, 9B active parameters; throughput 27344.4444444 tokens/s (architecture-class calibration, 29 Oct 2026, efficient-engine prefill; capped at 100,000 tokens/s). The row's own scored cost basis: 1,075,415 input + 0 output tokens over 1,489 decisions. Generated tokens are counted at the same throughput as prefill. GPU-time cost USD 0.0159681353639466/1,000; old list-price estimate USD 0.0722239758226998/1,000 (qwen/qwen3.5-9b at USD 0.1/M input and USD 0/M output). Applied min(old list-price estimate, GPU-time cost) = USD 0.0159681353639466/1,000. Estimated, not a bill. USD 0.0159681354 per 1,000 decisions on the frozen 1,489-item common set of this draw (USD 0.0159238486 over all 1,500 items).
p50 latency: 0.3 s
Self-hosted latency estimate under the shared method (assumption, not measured)
Measured 30 Oct 2026 on an evaluator-owned Verda VM in FIN-03 (Finland, EU), one NVIDIA RTX PRO 6000 Blackwell Server Edition, 97,887 MiB reported, driver 580.178.04, image 24.04.cuda13.0. The two Piccolo models used the same VM sequentially. A first-party loopback HTTP shim invoked the pinned compiler and layout at 3e5e54b1554ad6ad85b46980719d6662a5980fd7, with local hash-verified full model files, read-only mounts, empty credentials, offline flags and network cut before sealed input. Python 3.12.12, torch 2.14.1, CUDA 13.0, transformers 5.12.1 and tokenizers 0.22.2 were pinned. FLA and causal-conv1d were absent; native Torch fallbacks were used. Public parity compared complete token IDs, candidates, keys, metadata and repetition behavior against the native compiler. Eighteen public parity cases and the independent golden fixture passed. Fresh timed short and 3,599-token probes followed separate warm-ups. Noul, choice, score, 255-choice, Unicode JSON, multi-question and repetition-boundary probes passed. An 84,389-token synthetic overflow returned HTTP 422 with zero usage. These are synthetic observations, not scored percentiles or full-ceiling capacity proof. A separate pre-warm passed. The complete 1,500-request run took 165.3157308101654 s; raw output sha256 9a3d964c7dec99faf20a8a68a98ce90f8bde223f086e9beebef72163f4f1c3bc was independently verified. Post-run health was ready; provider closure and shredding of temporary node and local copies were confirmed with matching retained checksums. GPU rental spend is separate from the benchmark's model-cost estimate.
Measured latency over 1,500 decisions: p50 0.094 s · p95 0.187 s
0 decisions over 10 s.
A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.