jiwo 0.8b (Qwen3.5-0.8B backbone with a trained decision head, one forward pass, native probabilities, self-hosted)
Data: JevBench v1.6.40, published 2026-10-22
Capability 47.2 · rank #97 of 139 on the open-weights board
Last measured: 2026-10-10 · measurement release: v1.6.9
Apache-2.0 (weights, model card); MIT (code, github.com/jiwidi/jiwo)
intelligence
15.9
calibration
78.5
speed
94.3
cost
78.4
Composite (secondary)
4.1 · Composite rank #144
Cost and measurement conditions
$0.0052 per 1,000 decisions (estimated)
ESTIMATE (*est.), exact base-model market reference; self-hosted open weights with no per-token tariff of their own. The unmodified frozen pricing_v15 resolves the author's own declared base spelling Qwen/Qwen3.5-0.8B straight through to deepinfra:Qwen/Qwen3.5-0.8B at USD 0.01 per 1M input and USD 0.05 per 1M output in the dated 25 Sep 2026 snapshot. No new release-lane pricing line. OUTPUT TOKENS ARE 0 BY CONSTRUCTION, not by estimate: the readout is a distribution over the item's own candidate labels from one forward pass and nothing is generated, so the row's cost rests on the input rate alone. This row's own usage on draw v1.6-regular-20261010-a2: 786,993 input and 0 output tokens over 1,500 decisions. Estimated, not charged.
Measured p50 latency: 0.020 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.19 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.020 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.19 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
The author's own install line from their model card (pip install 'jiwo[cuda] @ git+https://github.com/jiwidi/jiwo@8716abab') and their own `jiwo serve`, nothing patched. The server reads JIWO_CHECKPOINT and calls from_pretrained with NO revision and there is no JIWO_REVISION setting, so it was pointed at the LOCAL PINNED SNAPSHOT DIRECTORY and the served revision cannot have drifted to main. JIWO_MAX_LENGTH=65536, the author's own documented value for their published runs: the 8,192 default refuses longer prompts with HTTP 400 and never truncates. The trained context is 8,192 tokens, so the longest items of this draw are answered outside the range the model saw in training. CUDA graphs are OFF (JIWO_CUDA_GRAPHS unset): the author's own card says graphs agree with eager on only 99.7 % of fields, so eager is the exact path and the author's 14.5 ms headline uses a mode this measurement does not use. Requests are serialised by the author's own asyncio.Lock. evaluator-owned Lium RTX 5090 32 GB pod (pod calm-wolf-31, USD 0.72/h; Brasov, Romania, EU by the RIPE whois of its public IP, receipt r90/receipts/POD-IP-WHOIS-R90.txt); pod-level iptables egress cut installed, verified and TIMESTAMPED pod-side (logs/EGRESS-CUT-R90.log) BEFORE the sealed input was uploaded, so the ordering is provable from pod evidence and not only from the session transcript; HF/transformers offline, no credentials in the environment, one decision per request, no batching, no retry. The five rows ran strictly one at a time on the one card, each server started with setsid and stopped by its exact process group with a card-free check in between.