Chon-1.0-12B-Fastv1.7 · 10 s rule
Data: JevBench v1.7.1, published 2026-10-25 · method v1.7 (10 s rule), measured under the rule
Capability 70.1 · rank #30 of 159 on the open-weights board
Last measured: 2026-10-25 · measurement release: v1.7.1
Apache-2.0 (adapter); base google/gemma-4-12B-it Apache-2.0; serving package xoai/chon Apache-2.0
intelligence
60.3
calibration
79.9
speed
83.7
cost
58.7
Composite (secondary)
68.9 · Composite rank #17
Cost and measurement conditions
$0.024 per 1,000 decisions (estimated)
Estimated, not charged, with the same frozen sibling reference as Chon-1.0-12B. Exact base google/gemma-4-12B-it is explicitly unlisted by pricing module aa2afcd5 and returns no reference. The unchanged explicit reference is nearest listed sibling google/gemma-3-12b-it at USD 0.05/M input and USD 0.15/M output under frozen 25 September 2026 snapshots, as for other Gemma 12B rows. All-item native usage: 798841 input tokens, zero output, no think pass. Common 1485-item cost basis uses 707101 input tokens, excluding the same 15 long positions for every token-priced row's cost only; all 1500 answers count for capability. Cost on the common basis: USD 0.023808114478 per 1000 decisions; official cap USD 0.06459465517241379.
Measured p50 latency: 0.175 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.50 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.175 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.50 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
Evaluator-owned rented Verda RTX A6000 48 GB, FIN-01 (Finland, EU), non-preemptible single-GPU pod, 25 Oct 2026 00:24:58-00:49:47Z. Native author's chon.serve at 127.0.0.1:8000 with explicit local base and adapter, empty credential environment, read-only pinned mounts, --network none and host egress cut proven before sealed input; no vLLM or think pass. Input sha256 dfd36720077ef7c41ace367524896507c7f44ce71306fb3e659e8b85069d995b. Host validator PASS, 1500/1500 native HTTP 200; mean latency 0.219292 s. Node shred (0 files left) and exact instance and volume teardown verified. About USD 0.41 rental of USD 0.98 booking is separate from estimated inference tariff.
Measured latency over 1,500 decisions: p50 0.175 s · p95 0.348 s
0 decisions over 10 s.
A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.
Model and serving disclosure
Author's own xoai/chon v1.1.0 at 3fc64c0aa3b02cf234acc16c397bd2ce70b34df4; independently source-reviewed before execution; the Fast code gate and pre-registration amendments 1-2 apply. Adapter ninhnv/chon-1.0-12b-fast at 92c960ef1245f604642538c562ed9dec89bab48f, base google/gemma-4-12B-it at 707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7 (evaluator-selected; training revision unproven). Python 3.12, torch 2.9.1+cu128, transformers 5.20.2, peft 0.18.1, BF16 CUDA SDPA, default adapter merge/unload, no vLLM and no think pass. Image sha256:4659ede5b4a66983cb113724bb5eac662f2fa1f2ec9bb11fbb2a9e06dcb77751. Dependency lock identical to attempt 2 except transitive starlette 1.9.0 to 1.10.0. Frozen run_v15.py and typesafe adapter unchanged. Same author, base and identical adapter weight/calibration as the Chon-1.0-12B family; Fast disables thinking. PEFT LoRA r64, native choice/noul/score letter readout and author calibration: temperatures choice 1.0181, noul 0.3915, score 0.7489; post-temperatures 0.8/2.5/1.0; generic noul commit band [0.199,0.801]. Submitted max_state_tokens=6000 and truncate=true, silently retaining beginning plus omission marker for ordinary text/JSON, recent conversation turns or tail of a single huge last message; state limit excludes prompt scaffold. Counts from the sealed inference: 15 truncated states, all answered; 0 grouped (>16-option) decisions. Zero output tokens on every answer, no think pass, no vLLM. 1500/1500 native HTTP 200, zero refusals, no dropped/repaired/retried items. One self-hosted use of v1.6-regular-20261012 under a zero-refusal custody policy; the second and last admitted Chon-12B family use of this draw. ATTEMPT HISTORY: two initial image-create HTTP 400 refusals created no instance (Amendment 1). Actual attempt 1 on FIN-02 never obtained SSH; attempt 2 on FIN-01 stopped before sealed input because compose_proof.py shadowed hostcut() (UnboundLocalError); both stopped before any sealed byte. Amendment 2 admits only the host-cut variable fix and a third attempt, and raises the run envelope from USD 2 to USD 3 (owner-approved), with no ledger reset. Attempt 3 is this successful measurement. Previous attempts are receipts only, never scored or merged.