NVIDIA Nemotron 3.5 Super VLv1.7 · 10 s rule
Data: JevBench v1.8.7, published 2026-11-03 · method v1.7 (10 s rule), measured under the rule
Capability 59.6 · API offering, listed but not ranked on the API leaderboard
Measured on the 27 Oct 2026 draw, which is not yet equated to the reference scale; stronger anchor runs for that draw are being measured. Shown unranked until then.
Last measured: 2026-11-03 · measurement release: v1.8.7
Hosted API through OpenRouter; the weights licence is not assessed here (see the model page).
intelligence
60.4
calibration
58.8
speed
82.0
cost
47.2
Composite (secondary)
53.3 · Not ranked on this board
Cost and measurement conditions
$0.057 per 1,000 decisions (public list price)
Published OpenRouter tariff for the Crusoe bf16 endpoint of nvidia/nemotron-3.5-super-vl: USD 0.10 per million input tokens and USD 0.40 per million output tokens (fetched 2026-11-03T05:45:33Z; frozen pricing module sha256 2f52370fe1dba73d75a770a386df350cc76a0898304aeba9e29c91f6e4d3738b), counted from day 1 as the operator launch list price under the standing launch-list-price interpretation (I-1; the authority is recorded in the release proof). Hosted list price, so no GPU-time estimate applies. Cost = tariff x the engine-reported usage on the frozen 1,489-item common set of this draw: 607,138 input and 62,072 output tokens = 449.435863 tokens/decision. COST FIGURES: USD 0.057450 per 1,000 decisions. Cost on the 1,489 items every row on this draw answered (same-items rule); all-items cost USD 0.084429 per 1,000 decisions over all 1,500 decisions (1,015,566 input and 62,716 output tokens, reasoning disabled so no reasoning tokens). Tariff-based figure from engine-reported usage, not a bill.
p50 latency: 0.6 s
none (API)
Hosted API through OpenRouter, served by Crusoe (bf16): provider order Crusoe only, fallbacks off, bf16 quantization only. Every one of the 1,500 responses was checked for provider Crusoe and the exact model id nvidia/nemotron-3.5-super-vl. Reasoning disabled (reasoning.enabled=false). Strict json_schema output, temperature 0, max_tokens 4096, with the shared calibration system prompt: "You are a calibration engine. You never answer in prose. You output only a JSON object with the key 'probabilities' mapping every given option to a probability, all options included, values in [0,1], summing to 1." Latency is hosted-API latency measured from our server in Germany over OpenRouter (network round trip included): p50 0.63 s, p95 1.01 s, maximum 31.2 s; 2 decisions took longer than 10 s. 1,500 of 1,500 requests answered (HTTP 200). 56 upstream rate-limit responses (HTTP 429) were re-sent and are not counted as answers.
Measured latency over 1,500 decisions: p50 0.627 s · p95 1.011 s
2 decisions over 10 s (2 of them answered without error, counted as wrong).
A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.
Model and serving disclosure
FIRST MEASUREMENT of NVIDIA Nemotron 3.5 Super VL on the text JevBench (the image benchmark is a separate lane), as a hosted API with no weights pin. REASONING DISABLED: reasoning.enabled=false, as required by the holder of the sealed items for this use; a reasoning-on configuration could score differently. SERVING: OpenRouter, provider Crusoe (bf16), provider order Crusoe only, fallbacks off; all 1,500 responses were checked for provider Crusoe and the exact model id. Strict json_schema output, temperature 0, max_tokens 4096, and the shared calibration system prompt (quoted in the measurement conditions). LATENCY: hosted-API latency from our server in Germany over OpenRouter (network round trip included): p50 0.63 s, p95 1.01 s, maximum 31.2 s; 56 HTTP 429 upstream rate-limit responses were re-sent and are not counted as answers. TOKENS: engine-reported usage, 1,015,566 input and 62,716 output tokens over all 1,500 decisions (no reasoning tokens). COST: the published OpenRouter tariff for this endpoint, USD 0.10 per million input tokens and USD 0.40 per million output tokens, counted from day 1 as a launch list price; no GPU-time estimate applies. DRAW: measured on the 27 Oct 2026 regular draw (fresh sealed items; not yet equated to the reference scale, so its numbers are shown as published, offset 0). This draw has no subject-topic labels yet, so the row has no topic cells; its use-case and language cells are measured on its own draw.