Krino-2B
Data: JevBench v1.6.40, published 2026-10-22
Capability 42.4 · rank #108 of 139 on the open-weights board
Last measured: 2026-10-12 · measurement release: v1.6.10
Apache-2.0 (adapter, head and code; Kev Apache-2.0; base Qwen3.5-2B-Base Apache-2.0)
intelligence
22.0
calibration
62.8
speed
91.9
cost
61.0
Composite (secondary)
8.7 · Composite rank #136
Cost and measurement conditions
$0.020 per 1,000 decisions (estimated)
v1.5-M2 base-model reference: frozen DeepInfra snapshot jevbench-v15-measure-20260925 receipts/DEEPINFRA-MODELS-20260925.json (SHA-256 84eb7926acd4e93b71d695be24898e2ca999b20011d1b66cc0348ce7c6c3546f); listing Qwen/Qwen3.5-2B, USD 0.02/M input and USD 0.10/M output. The released checkpoint names Qwen/Qwen3.5-2B-Base; the frozen DeepInfra listing omits the -Base suffix. The requester cited the Qwen3.5-4B USD 0.03/M input reference, but the matching 2B family listing is used here (precedent classone-qwen3.5-2b). Self-hosted GPU deployment has no provider tariff, so price_kind is estimate. Kev generates no free-form text. However, kev/api.py output_tokens() tokenizes JSON-serialized answers, so reported usage.output_tokens is nonzero for normal answers; it is not zero by construction. COMMON COST BASIS (v1.6.30): cost per 1,000 decisions is computed on the common 1,485 items of this draw, leaving out the 15 very long items (at least 76,358 input tokens as counted by the rows that answered them). They are the 15 items the Jev reference refused with max_tokens_exceeded, which the other rows that did not answer them refused or failed on; their tokens count for no row, whether the row refused or answered them, and a refused item still counts as a wrong answer for Intelligence. This row: USD 0.019952 per 1,000 decisions (0.309x the official cost cap USD 0.064595), against USD 0.032978 over all 1,500 items (v1.6.29).
Measured p50 latency: 0.030 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.21 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.030 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.21 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
evaluator-owned Lium GPU pod (H100), offline read-only container (--network none); author's `python3 -m krino.serve` (Kev @fe64b127 serve path, documented per-type temperature map choice 1.95 / noul 0.30 / score 0.50 from krino.json), KEV_TRUNCATE_STATES=1, reference kernels (KEV_FUSED=0: the documented pip install has no flash-linear-attention) Paid fast-lane order 8c7dbe11 (public). Kev architecture (LoRA r16 + pointer head) on Qwen/Qwen3.5-2B-Base @b1485b2f. Runtime: reviewed pytorch 2.11 base + Kev uv.lock closure (torch 2.8.0) installed offline from SHA-256-pinned wheels (PR #335). Kev reports usage.output_tokens as the token count of the serialized answer JSON (nothing is generated); priced as reported at the reference output rate.