Vansa-3.4 (Vansa, hosted System One API)v1.7 · 10 s rule

Data: JevBench v1.8.7, published 2026-11-03 · method v1.7 (10 s rule), re-scored from the recorded per-decision times

Capability 63.7 · API offering, ranked on the API leaderboard · Capability rank #26 of 33 within the Jev-class caps

Measured 2026-10-06 on A4 ∪ P, 600 items, equated (+2.57 I / +3.35 C); round score-a4-2.

proprietary hosted API

Base model: Qwen3.5-4B (self-reported) · Developer-reported to Benchmark Heaven (private correspondence, 1 Oct 2026): built on Qwen3.5-4B via an open decision fine-tune, with Vansa's own adapters; the intermediate fine-tune is not named. Not publicly documented or independently verified.

intelligence

40.9

calibration

86.5

speed

95.0

cost

61.9

Composite (secondary)

42.6 · Composite rank #29 of 48 on the API leaderboard

Cost and measurement conditions

$0.019 per 1,000 decisions (public tariff)

measured: token usage of the 6 Oct 2026 A4 u P run x tariff USD 0.034/M input, 0/M output (spec vansa-3.4.json), per 1,000 decisions on the v1.6.1 common cost basis (591 of 600 items; the 9 A4 u P items outside the Jev reference's accepted input range are excluded, cost-basis-a4.json)

p50 latency: 0.2 s

none (API)

Measured latency over 600 decisions: p50 0.158 s · p95 0.198 s

0 decisions over 10 s.

A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.

JevBench v1.6.43 values (method v1.6, before the 10 s rule)

Capability 63.68 · Composite 42.60 · rank #75 on the all-systems board · #27 API

intelligence 40.85 · calibration 86.52 · speed 95.49 · cost 61.92

v1.6 speed basis (124 items): p50 0.150 s · p95 0.189 s

These figures are kept for comparison. The current figures above are re-scored under the 10 s rule from the recorded per-decision times and replace them.

Breakdowns (not part of the score)

Chance-corrected competence (0 = chance, 100 = perfect) on A4 ∪ P (600 items: 300 open + 300 sealed; fewer sealed items than the full set, so wider uncertainty). Raw values, not equated. Category cells use A4+P+L1+L2+L3; per-type/tier cells use the original basis above. Radars against Jev.

Request typeOpenSealed
choice62.960.2
noul32.821.3
score34.418.1

Capability by subject topic

Rules, policy & law26.1n=1133
Coding & software40.9n=202
Math & numbers0.0n=269
Finance & commerce33.9n=178
Support & operations38.8n=109
Everyday language55.1n=130
Safety & security43.1n=84

Use cases (TypeSafe categories)

Model routing53.6n=267
Legal & compliance40.3n=269
Customer support30.9n=156
E-commerce32.2n=88
Insurance claims28.2n=92
Risk assessment26.6n=88
Financial crime7.2n=76
Feature extraction11.4n=90
Lead generation43.5n=86
Recruiting1.7n=88
Knowledge graphs3.3n=82
LLM guardrails39.9n=82
Moderation42.0n=69
Code linting7.2n=81
Search & retrieval46.8n=84
Science0.0n=82
Advertising21.8n=85
Gaming0.0n=79
Demand forecasting0.0n=72

Cells need at least 15 answered items; † fewer than 30 (indicative only, listed below the radar in the compare view).

Previous measurement (v1.5.x, not comparable)

Capability 63.7 · composite 42.6 · cost $0.019 per 1,000 decisions (public list price)

Last measured 2026-10-06, measurement release v1.6.1. Older method and scale; the current figures above replace it.

Published source