DeepSeek V4.1 Flash (thinking default)v1.7 · 10 s rule
Data: JevBench v1.8.7, published 2026-11-03 · method v1.7 (10 s rule), re-scored from the recorded per-decision times
Capability 89.6 · API offering, ranked on the API leaderboard · outside the Jev-class caps (cost 14.9× Jev (v1.5 reference); latency 3.6× Jev (v1.5 reference)), below the Capability ranking
Measured 2026-10-06 on A4 ∪ P, 600 items, equated (+2.57 I / +3.35 C); round score-a4-2.
open weights, proprietary API route
intelligence
84.5
calibration
94.8
speed
66.0
cost
19.6
Composite (secondary)
6.9 · Composite rank #43 of 48 on the API leaderboard
Cost and measurement conditions
$0.480 per 1,000 decisions (public tariff)
measured: token usage of the 6 Oct 2026 A4 u P run x tariff USD 0.15/M input, 0.6/M output (spec deepseek-flash.json), per 1,000 decisions on the v1.6.1 common cost basis (591 of 600 items; the 9 A4 u P items outside the Jev reference's accepted input range are excluded, cost-basis-a4.json)
p50 latency: 2.2 s
none (API)
Measured latency over 600 decisions: p50 2.205 s · p95 11.414 s
34 decisions over 10 s (31 of them answered without error, counted as wrong).
A decision that takes longer than 10 seconds (measured time) counts as a wrong answer and as zero calibration for that item: each type's calibration is multiplied by the share of its decisions answered within 10 s. The limit uses the measured time (raw). The displayed adjusted time for self-hosted rows (2 × + 0.15 s) is an assumption and is not used for the limit.
JevBench v1.6.43 values (method v1.6, before the 10 s rule)
Capability 97.35 · Composite 7.09 · rank #180 on the all-systems board · #40 API
intelligence 94.71 · calibration 100.00 · speed 68.07 · cost 19.56
v1.6 speed basis (124 items): p50 2.038 s · p95 7.648 s
These figures are kept for comparison. The current figures above are re-scored under the 10 s rule from the recorded per-decision times and replace them.
Breakdowns (not part of the score)
Chance-corrected competence (0 = chance, 100 = perfect) on A4 ∪ P (600 items: 300 open + 300 sealed; fewer sealed items than the full set, so wider uncertainty). Raw values, not equated. Category cells use A4+P+L1+L2+L3; per-type/tier cells use the original basis above. Radars against Jev.
| Request type | Open | Sealed |
|---|---|---|
| choice | 95.5 | 96.1 |
| noul | 93.5 | 87.9 |
| score | 58.0 | 60.5 |
Capability by subject topic
| Rules, policy & law | 80.7 | n=1133 |
|---|---|---|
| Coding & software | 67.7 | n=202 |
| Math & numbers | 67.0 | n=269 |
| Finance & commerce | 84.0 | n=178 |
| Support & operations | 77.3 | n=109 |
| Everyday language | 90.0 | n=130 |
| Safety & security | 93.6 | n=84 |
Use cases (TypeSafe categories)
| Model routing | 81.0 | n=267 |
|---|---|---|
| Legal & compliance | 67.7 | n=269 |
| Customer support | 87.8 | n=156 |
| E-commerce | 80.2 | n=88 |
| Insurance claims | 90.8 | n=92 |
| Risk assessment | 94.5 | n=88 |
| Financial crime | 74.8 | n=76 |
| Feature extraction | 81.3 | n=90 |
| Lead generation | 79.4 | n=86 |
| Recruiting | 77.8 | n=88 |
| Knowledge graphs | 83.1 | n=82 |
| LLM guardrails | 95.5 | n=82 |
| Moderation | 76.0 | n=69 |
| Code linting | 70.1 | n=81 |
| Search & retrieval | 91.1 | n=84 |
| Science | 87.7 | n=82 |
| Advertising | 77.1 | n=85 |
| Gaming | 70.0 | n=79 |
| Demand forecasting | 65.4 | n=72 |
Cells need at least 15 answered items; † fewer than 30 (indicative only, listed below the radar in the compare view).
Previous measurement (v1.5.x, not comparable)
Capability 89.6 · composite 6.9 · cost $0.480 per 1,000 decisions (public list price)
Last measured 2026-10-06, measurement release v1.6.1. Older method and scale; the current figures above replace it.