modernbert-jev v2
Data: JevBench v1.6.40, published 2026-10-22
Capability 34.6 · Not ranked in the current release
latency 8.7× Jev (v1.5 reference)
Last measured: 2026-10-10 · measurement release: v1.6.8
Apache-2.0 (model card; the code README and pyproject.toml also declare Apache-2.0, without a separate LICENSE file)
intelligence
2.6
calibration
66.7
speed
57.8
cost
70.4
Composite (secondary)
0.0 · Composite rank #184
Cost and measurement conditions
$0.0097 per 1,000 decisions (estimated)
ESTIMATE (*est.), size-class reference; self-hosted open weights, no public tariff. The same choice the published quyet-1-0-small-en row carries for the identical ModernBERT-base backbone: DeepInfra base-size encoders at USD 0.005 per 1M input and 0 output (deepinfra:thenlper/gte-base, BAAI/bge-base-en-v1.5, intfloat/e5-base-v2 and sentence-transformers/all-mpnet-base-v2 are all 5e-7 cents per input token in the frozen 25 Sep 2026 DeepInfra snapshot). Output tokens are a literal 0 reported by the author's server (NLI entailment read-out, nothing is generated). This row's own usage on draw v1.6-regular-20261010-a2: 2,916,019 input and 0 output tokens over 1,500 decisions. Estimated, not charged.
Measured p50 latency: 2.600 s (on-CPU, no network, loopback; not directly comparable with API latency, which we time end to end from our measuring host, network included)
Ranking and cap use: 5.35 s (est. incl. network (2x + 150 ms))
Self-hosted model: measured p50 2.600 s on-CPU, no network (timed on our CPU host over loopback). Not directly comparable with API latency, which we time end to end from our measuring host, network included. The ranking and latency cap use the same estimate for every self-hosted row: 2 × measured + 150 ms = 5.35 s. Model authors can ask us to time a hosted endpoint instead (on request).
x2 + 0.15 s (assumption, not measured)
Sandy, the evaluator's own shared CPU host (AMD Ryzen 5 3600, 6 cores / 12 threads, Zen 2, AVX2), NOT a dedicated machine: other jobs ran beside it and the 1-minute load average was logged every 60 s. The author's own CLI at github.com/KSonny4/modernbert-jev@b1866e4d, `modernbert-jev serve --model <pinned snapshot dir>` at its own defaults (1 ONNX Runtime thread, 127.0.0.1), inside a loopback-only network namespace with an outbound probe that failed as required, no credentials in the environment. int8 (unsigned) ONNX as shipped. Speed is a lower bound for this model on a dedicated core.