Image JevBench v0.3.0 · individual system

Jev 1.14.0 + images, preview (TypeSafe)

Closed APIBase model: undisclosed

Published source

the operator's hosted API

Image JevBench v0.3.0 composite score

24.376

Rank #16 of 56 ranked systems.

From the published full-benchmark aggregate. The composite combines intelligence, calibration, speed and cost.

Published axes

intelligence
49.9
calibration
70.6
speed
84.6
cost
33.9

Cost and speed evidence

Cost per 1,000 decisions
USD 0.159185
Cost basis
TypeSafe list price USD 0.042 per 1M input tokens x usage.input_tokens (images billed as input tokens)
Measured latency · p50 / p95
0.393 s / 0.875 s
Adjusted latency · p50 / p95
0.393 s / 0.875 s
Latency adjustment
none (hosted API, measured as is)
Public accuracy
61.3%
Sealed accuracy
70.5%
Measurement note
Preview feature: TypeSafe documents image input as "Dreamer-ready", not yet production; the request format, limits and accuracy can change between releases. Official ImageJev v0.3 A300+P300 pass 1 on 14 Oct 2026 (22:05-22:13 UTC), Jev 1.14.0 (TypeSafe's newest release, available since 14 Oct 2026, alias jev-preview) with images. 598 of 600 answered; 2 invalid decisions count wrong. Request: the item question and image as state, one choice question with the options as criteria (TypeSafe-native; not the lettered v0.2 prompt), native option probabilities, no retries; failed requests count wrong at cost 0. Images outside TypeSafe's limits were rescaled (Lanczos, aspect ratio kept; 172 images) or, for 7 BMP files, re-encoded as PNG: 179 of the 600 images (99 public, 80 sealed); all others were sent unchanged. Other systems received images under their own provider limits (for example Wity: 12 MB request body), so image treatment differs between rows. The API rejects any image side below 280 px although the per-side minimum in TypeSafe's docs read 256 px when we tested (the docs now also state a 78,400-pixel minimum). An earlier Jev 1.14.0 attempt was invalidated for that reason (our adapter had scaled small images to 256 px; 27 requests failed with HTTP 400) and is excluded; no result of it is used. Exposure was logged before dispatch. Before dispatch the adapter author read the sealed images' pixel dimensions only (no content) outside the runner to count limit violations.

Published 2026-10-15. Scores and ranks can change in a later release. See the method notes.