Arbiter-4B (Goddragon, NeoHorse 4B pointer model, author server, self-hosted)
Data: JevBench v1.6.39, published 2026-10-22
Capability 52.3 · rank #88 of 139 on the open-weights board
Last measured: 2026-10-21 · measurement release: v1.6.36
Apache-2.0 (HF metadata and LICENSE), public, not gated
intelligence
27.4
calibration
77.2
speed
92.2
cost
66.2
Composite (secondary)
15.9 · Composite rank #116
Cost and measurement conditions
$0.013 per 1,000 decisions (estimated)
ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module (sha256 1f4d1f7b1ce97ab27b938043444b9c2757e4a9dbecc799a803122588401f5bf1, 20 Oct additive re-freeze; this base resolves identically under the earlier 6202ef78...): base string Qwen/Qwen3.5-4B (the authors' stated Qwen3.5-4B / NeoHorse lineage, unverified) priced as Qwen/Qwen3.5-4B -> deepinfra:Qwen/Qwen3.5-4B USD 0.030000000000000002/M input, USD 0.15/M output. Estimated, not charged, not a quote from the author or any host. Tokens as reported by the author server's own usage block: 662,682 input on the 1,485 answered items (equal to our offline count through the author's encode() item by item, pre-registered), 0 output tokens (one forward pass with a pointer readout, no generation); the 15 refused items count 0 tokens. COMMON COST BASIS (v1.6.30): cost per 1,000 decisions is computed on the common 1,485 items of this draw, leaving out the 15 very long items (at least 76,358 input tokens as counted by the rows that answered them in full), which this author's server refuses; their tokens count for no row's cost.
Measured p50 latency: 0.047 s (on-GPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 0.24 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 0.047 s on-GPU, no network (timed on our GPU over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 0.24 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
Evaluator-owned rented Verda GPU VM, 1x NVIDIA A100-SXM4 80 GB (FIN-02, Finland, EU, pay-as-you-go non-spot, USD 2.168/h), shared by the two NeoHorse-family rows as two separate back-to-back uses (sealed input deleted from the node between them). The author's own server.py as shipped on stock PyPI vLLM 0.30.0 / torch 2.13.0 in a pre-built image, container with --network none (cut proven from inside before the sealed input), Hub offline, every credential unset, read-only bundle, all weight shards sha256-verified; driven by the unchanged pinned evaluator driver, one request at a time. The standard self-hosted x2 + 0.15 s adjustment applies. Measured 21 Oct 2026 on the frozen v1.6-regular-20261012 self-hosted input (sha256 dfd36720...); determinism 50/50 bit-identical; VM and volume destroyed afterwards.