Chon-1.0-12B
Data: JevBench v1.6.39, published 2026-10-22
Capability 85.3 · rank #1 of 139 on the open-weights board
Last measured: 2026-10-19 · measurement release: v1.6.32
Apache-2.0 (adapter); base google/gemma-4-12B-it Apache-2.0; serving package xoai/chon Apache-2.0
intelligence
79.9
calibration
90.8
speed
65.5
cost
49.0
Composite (secondary)
64.8 · Composite rank #27
Cost and measurement conditions
$0.050 per 1,000 decisions (estimated)
ESTIMATE: base-model reference price per v1.5-M2 section 2 with INTERPRETATION-1 I-3 (frozen 25 Sep 2026 cut-off snapshots). The exact base google/gemma-4-12B-it (adapter_config base_model_name_or_path, chon_config base_model; pinned commit 707f0a3b) has no listing on OpenRouter or DeepInfra in those snapshots (OPENROUTER-MODELS-20260925.json SHA-256 f5d0cd8e0f7d2d6514748c99bfcaeacc09775bac7aa7eecf765af4ba04b0dcd5; DEEPINFRA-MODELS-20260925.json SHA-256 84eb7926acd4e93b71d695be24898e2ca999b20011d1b66cc0348ce7c6c3546f) nor in the order's OpenRouter snapshot read 2026-10-19T03:50:11Z, so the nearest listed sibling of the same family and size, OpenRouter google/gemma-3-12b-it, is used: USD 0.05/M input, USD 0.15/M output (DeepInfra lists the same). Same basis as the Cygnet, Winnow-12B, Jev-Omni, Torchcast-Decision-12B and Metask-12B rows. Applied to the server's own usage: input tokens encoded and output tokens generated by its think-when-unsure pass (up to 512 per decision). Self-hosted, no bookable tariff; estimated, not charged. COMMON COST BASIS: published cost is recomputed over the same remaining 1,485 items as every other row; the 15 long states are answered and billed too. All 1,500 items count for Intelligence.
Measured p50 latency: 0.134 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.42 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.134 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.42 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
evaluator-owned Verda GPU VM (RTX 6000 Ada), offline read-only container (--network none); author's chon.serve unchanged (xoai/chon @09ab5fa4) with the adapter's chon_config.json (bf16 LoRA r64 on the pinned google/gemma-4-12B-it @707f0a3b, letter readout, per-type calibration, yes/no commit rule, think-when-unsure pass up to 512 tokens by the base model for at most 45 % of questions) on the documented stack torch 2.9.1+cu128 / transformers 5.20.1 / peft 0.18.1