JevBench v1.6.16 · one row added

JevBench · vLLM-SR d3 joins the regular draw

The ten full-draw rows of v1.6.15 retain their full 1,500-item results, point scores and categories on draw v1.6-regular-20261012; reflex-instinct-0.6b stays listed but not ranked (its Speed figure is not official), and the unranked Microsoft A2 re-run is carried through unchanged. One self-hosted system is added: vLLM-SR d3 (vllm-sr/d3, a Qwen3.8-27B fine-tune, Apache-2.0 weights, training data not disclosed). It has the highest Capability of this draw, but its cost is an estimate far above the Jev-class cost cap, so it is not on the default capped Capability ranking and the squared cost gate keeps its composite small. It was timed on an RTX PRO 6000 GPU, while this draw’s other GPU rows, Perplexity Decider and Torchcast-Decision-27B were timed on an H100. Ranks recompute over the new roster; no earlier point score moves.

Aggregate results JSON · SHA-256 8bb77523f5f430d2511c80934b465f26d163dc9d4870095be1c8520f78883bbe · Previous v1.6.15 · Full historical board

JevBench v1.6.16 · headline

JevBench Capability Score

Capability ranking of Jev-class systems

Capability Score averages Intelligence and Calibration. metask-jev-rain-31B-trained leads the Jev-class systems with 78.9.

Jev-class means at most 2× the cost and median latency of Jev (v1.5 reference). How we choose ↘

Adjust cost / latency caps · 2× official
Official
#SystemICCap.$/1k
  1. 1metask-jev-rain-31B-trained70.749.278.9$0.049*
  2. 2metask-jev-rain-12B-trained67.851.577.9$0.041*
  3. –Microsoft-Decision-1API59.262.272.2$0.018PRELIMINARY — API supplement A2 (300 sealed + 300 public), not the S1200 full set; equating pool 4 of 6
  4. 3attest-31B-v1.0API65.250.969.5$0.043*
  5. 4Torchcast Decision 4B54.261.268.7$0.020*
  6. 5Uniphore-4B-Decider34.264.757.1$0.015*
  7. 6Standard One 3B SH32.848.855.6$0.051*
  8. 7Krino-2B22.054.542.4$0.033*
  9. 8Phocinae-Largha-150M-v13.695.330.2$0.0014*
  10. 9Julia-12.489.99.9$0.0022*

Wide coloured bar = Capability Score (0–100). Thin bars: cost above, median latency below; shared log ratio scale 0.25× … 64× Jev (v1.5 reference), ticks at 1× and 2× (cap). Shorter is cheaper or faster. * = est. (estimated cost). # counts ranked Jev-class systems; “–” marks unranked or outside systems. Tap ⓘ or hover a row or thin bar for details.

Cost and latency correlate here (Spearman ρ = -0.34, n = 11).

  • Closed API (weights not public) (1)
  • Open weights · LLM decoder (8)
  • Open weights · encoder / classifier (2)
  • green: ≤ reference
  • amber: ≤ cap (2× reference)
  • red: > cap
Show general-purpose LLMs and other systems outside the limits

Sorted by Capability Score, not numbered. Each row says which limit it misses, measured against Jev ($0.032 per 1,000 decisions, median 0.62 s).

  1. –vLLM-SR d373.717.983.0$0.55*Outside: cost 16.9× Jev (v1.5 reference)

Jev-class = cost per decision at most 2× Jev's (≤ $0.065 per 1,000 decisions) and median latency at most 2× Jev's (≤ 1.23 s), the adjusted p50 — the same median the speed chart plots, not the four-axis Speed score. 10 of 11 systems qualify; the other 1, including the general-purpose LLMs, are listed below the divider in the ranking. The charts below show speed and cost beside Capability Score; the official JevBench Score weighs all four axes.

Capability against cost and speed

Jev-class systems are shown by default. Bubble size follows the official JevBench Score; official rank stays unchanged. The five most capable Jev-class systems are labelled.

Capability vs cost

Upper right is best: more capable and cheaper. Cost is USD per 1,000 decisions on a log scale. The dashed line is 2× Jev’s cost.

Capability vs cost: 9 systems. Upper right is best: more capable and cheaper.102030405060708090100$0.0010$0.010$0.10$ per 1,000 decisions (log)Capability↑2× Jev (v1.5 reference)← priciercheaper →1. metask-jev-rain-31B-trained2. metask-jev-rain-12B-trained3. attest-31B-v1.04. Torchcast Decision 4B5. Uniphore-4B-Decider
9 systems. Tap a bubble for its values.

Capability vs speed

Upper right is best: more capable and faster. Speed here is the median-latency speed — the same adjusted median (p50) latency the Jev-class limit uses — a log scale, so each 20 points is 10× faster (median under the numbers). Self-hosted systems are plotted at the est. incl. network latency (2x measured + 150 ms); their measured loopback p50 (on-GPU, no network; not directly comparable with API latency) is in the tooltip. The dashed line is 2× Jev’s median latency.

Capability vs speed: 9 systems. Upper right is best: more capable and faster.010203040506070809010070≈3.2 s80≈1.0 s90≈316 ms100≈100 msMedian-latency speedCapability↑2× Jev (v1.5 reference)← slowerfaster →1. metask-jev-rain-31B-trained2. metask-jev-rain-12B-trained3. attest-31B-v1.04. Torchcast Decision 4B5. Uniphore-4B-Decider
9 systems. Tap a bubble for its values.
  • Open weights · LLM decoder (8)
  • Open weights · encoder / classifier (2)
  • faint = outside Jev-class
12 of 12 systems

Model kind

Jev-class

Release

No family reported in this release.

Parameters

No system in this release reports an exact parameter count.

Developer/API price $/1k
Base-model price $/1k
Scored cost $/1k
Alternative price $/1k

No system in this release reports an alternative pricing scenario.

p50 latency (s)
p95 latency (s)

JevBench v1.6.16

JevBench Composite Score: 10 ranked systems

Official· four axes 0–100, equal-weight harmonic mean · Method notes ↓

Weights:
Adjust weights ↓

View by:Capability ↑

The official order weighs Intelligence, Calibration, Speed and Cost equally. Each button re-sorts the same systems by one axis.

Greener = stronger within its column.

11 of 11 systems, sorted by official rank, #1 first.

  1. Microsoft-Decision-1 (preliminary)APInewPRELIMINARY — API supplement A2 (300 sealed + 300 public), not the S1200 full set; equating pool 4 of 6Closed APIBase model: Qwen3.5-9Bsource · Base model disclosed by Microsoft; the Decision-1 endpoint is evaluated as a hosted API offering.71.5I 59C 85S 89K 62$0.018
  2. 1Torchcast Decision 4Bnewfine-tune · 4BBase model: undisclosed69.6I 54C 83S 94K 61est.$0.020
  3. 2attest-31B-v1.0APInewbase · 31BBase model: undisclosed68.0I 65C 74S 97K 51est.$0.043
  4. 3metask-jev-rain-12B-trainednewadapter · 12B · W4A16 INT4Base model: undisclosed67.6I 68C 88S 73K 51est.$0.041
  5. 4metask-jev-rain-31B-trainednewadapter · 31B · W4A16 INT4Base model: undisclosed66.9I 71C 87S 83K 49est.$0.049
  6. 5Uniphore-4B-Decidernewfine-tune · 4BBase model: undisclosed27.2I 34C 80S 88K 65est.$0.015
  7. 6Standard One 3B SHnewfine-tune · 3BBase model: undisclosed21.7I 33C 78S 87K 49est.$0.051
  8. 7Krino-2Bnewadapter · 2BBase model: undisclosed8.5I 22C 63S 92K 54est.$0.033
  9. 8vLLM-SR d3newfine-tune · 26BBase model: undisclosed5.6I 74C 92S 87K 18est.$0.546
  10. 9Phocinae-Largha-150M-v1newfine-tune · 144MBase model: undisclosed0.1I 4C 57S 77K 95est.$0.0014
  11. 10Julia-1newfine-tune · 144MBase model: undisclosed0.0I 2C 17S 79K 90est.$0.0022
Weights:
Adjust weights ↓

Weights are relative: each axis counts in proportion to its slider. The score stays a weighted harmonic mean with the low-axis gates; the gates still apply when an axis sits at 0. Only equal weights give the official JevBench Score and rank.

Score = 4 / (1/I + 1/C + 1/S + 1/K) (each 0–100; × (axis / 50)² for Intelligence, Speed or Cost below 50)

  • Closed API (weights not public) (1)
  • Open weights · LLM decoder (8)
  • Open weights · encoder / classifier (2)
  • Shown, not ranked
I, C, S, K = Intelligence, Calibration, Speed, Cost; the est. pill = estimated cost; ann. = announced price; API = the operator's endpoint saw held-out benchmark inputs, without answers; $/1k decisions = US dollars per 1,000 decisions (not heat-shaded). Names link to each project.

Compare two systems

Pick any two. Radars for the score axes, use cases, chance-corrected competence per request type on the open and sealed sets, and competence per tier on each set. Further out is better on every spoke; the link keeps the pair.

The four score axes

Radar: the four score axes, two systemsThe four score axes, Torchcast Decision 4B vs attest-31B-v1.0. Intelligence: 54.2 vs 65.2; Calibration: 83.1 vs 73.8; Speed: 93.6 vs 97.1; Cost: 61.2 vs 50.9.50100Intelligence: Torchcast Decision 4B: 54.2; attest-31B-v1.0: 65.2Intelligence54.2 · 65.2Calibration: Torchcast Decision 4B: 83.1; attest-31B-v1.0: 73.8Calibration83.1 · 73.8Speed: Torchcast Decision 4B: 93.6; attest-31B-v1.0: 97.1Speed93.6 · 97.1Cost: Torchcast Decision 4B: 61.2; attest-31B-v1.0: 50.9Cost61.2 · 50.9
0–100, the values in the table. An axis a system has no published value for is left as a gap (it counts as 0 in the composite).

Use cases (TypeSafe categories)

Radar: use cases (typesafe categories), two systemsUse cases (TypeSafe categories), Torchcast Decision 4B vs attest-31B-v1.0. Model routing: 69.9 vs 82.4; Financial crime: 56.3 vs 51.1. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Model routing: Torchcast Decision 4B: 69.9; attest-31B-v1.0: 82.4 — Model routing: deciding which LLM or model tier should handle a prompt; classifying a request's intent, domain, difficulty or risk to pick a model. 217 items (75 open, 142 sealed).Model routing69.9 · 82.4Financial crime: Torchcast Decision 4B: 56.3; attest-31B-v1.0: 51.1 — Financial crime: suspicious transactions, KYC, fraud alerts, entity matching for investigations. 30 items (4 open, 26 sealed).Financialcrime56.3 · 51.1

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Measured application-category competence on this release’s item set. Chance-corrected competence per category (0 = chance, below chance negative, 100 = perfect), Torchcast Decision 4B: open+sealed; attest-31B-v1.0: open+sealed items pooled; only categories with at least 30 items are spokes, smaller ones are listed below. Hover a category for its definition and item count.A (Torchcast Decision 4B) is measured on 2 of 2 spokes only, so it is drawn as points, not a shape — measured on fewer items; full radar after the next re-measure.B (attest-31B-v1.0) is measured on 2 of 2 spokes only, so it is drawn as points, not a shape — measured on fewer items; full radar after the next re-measure.Gaps, not zeros: Open spokes (n/a or n=…) have fewer than 30 answered items for that system or no published value; hosted APIs answer a smaller item set, so more of their category cells stay under that.

Low sample, n < 30 — indicative only

Category (items)A: Torchcast Decision 4BB: attest-31B-v1.0
LLM guardrails (27)48.3 n=2778.8 n=27
Lead generation (25)53.8 n=2572.4 n=25
Search and retrieval (24)90.9 n=24100.0 n=24
Semantic code linting (24)34.2 n=2469.2 n=24
Scientific discovery (23)19.8 n=2359.2 n=23
Graphs and knowledge graphs (22)25.2 n=2279.3 n=22
Gaming (22)22.2 n=2231.1 n=22
Advertising (22)19.3 n=2241.0 n=22
Feature extraction for predictive modeling (19)0.0 n=1981.7 n=19
Risk assessment (18)23.7 n=1848.0 n=18
Recruiting (18)50.4 n=1857.8 n=18
Moderation and trust and safety (18)9.8 n=1892.1 n=18
E-commerce marketplaces (17)36.9 n=1759.6 n=17
Demand forecasting (17)29.7 n=1740.2 n=17
Insurance claims (15)25.0 n=1545.8 n=15

No published value for either system (under 15 answered items, or no per-category values): Customer support (14), Legal and compliance (10).

Not drawn on the radar: with so few items a single answer moves a category score by several points, so these values are noise-prone. These values become spokes once the item pool reaches 30 items per category.

What each category means · items per category
  • Model routing — deciding which LLM or model tier should handle a prompt; classifying a request's intent, domain, difficulty or risk to pick a model. 217 items (75 open / 142 sealed)
  • Financial crime — suspicious transactions, KYC, fraud alerts, entity matching for investigations. 30 items (4 open / 26 sealed)

Competence per request type, open / sealed

Radar: competence per request type, open / sealed, two systemsCompetence per request type, open / sealed, Torchcast Decision 4B vs attest-31B-v1.0. Choice · open: 75.5 vs 73.2; Choice · sealed: 62.8 vs 74.6; Noul · open: 61.9 vs 68.7; Noul · sealed: 50.6 vs 68.2; Score · open: 42.0 vs 52.9; Score · sealed: 32.5 vs 53.8. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Choice · open: Torchcast Decision 4B: 75.5; attest-31B-v1.0: 73.2Choice · open75.5 · 73.2Choice · sealed: Torchcast Decision 4B: 62.8; attest-31B-v1.0: 74.6Choice ·sealed62.8 · 74.6Noul · open: Torchcast Decision 4B: 61.9; attest-31B-v1.0: 68.7Noul · open61.9 · 68.7Noul · sealed: Torchcast Decision 4B: 50.6; attest-31B-v1.0: 68.2Noul · sealed50.6 · 68.2Score · open: Torchcast Decision 4B: 42.0; attest-31B-v1.0: 52.9Score · open42.0 · 52.9Score · sealed: Torchcast Decision 4B: 32.5; attest-31B-v1.0: 53.8Score ·sealed32.5 · 53.8

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Chance-corrected competence for Choice, Noul and Score on the 300 open and 1200 sealed decisions.

Competence per tier — open set

Radar: competence per tier — open set, two systemsCompetence per tier — open set, Torchcast Decision 4B vs attest-31B-v1.0. Easy: 89.0 vs 87.3; Standard: 65.3 vs 55.9; Judge: 53.2 vs 63.1; Hard: 59.7 vs 67.3. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Easy: Torchcast Decision 4B: 89.0; attest-31B-v1.0: 87.3Easy89.0 · 87.3Standard: Torchcast Decision 4B: 65.3; attest-31B-v1.0: 55.9Standard65.3 · 55.9Judge: Torchcast Decision 4B: 53.2; attest-31B-v1.0: 63.1Judge53.2 · 63.1Hard: Torchcast Decision 4B: 59.7; attest-31B-v1.0: 67.3Hard59.7 · 67.3

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Per-tier competence, the three request types pooled by their published decision counts.

Competence per tier — sealed set

Radar: competence per tier — sealed set, two systemsCompetence per tier — sealed set, Torchcast Decision 4B vs attest-31B-v1.0. Easy: 69.7 vs 82.2; Standard: 62.1 vs 69.7; Judge: 43.4 vs 67.4; Hard: 49.8 vs 63.9. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Easy: Torchcast Decision 4B: 69.7; attest-31B-v1.0: 82.2Easy69.7 · 82.2Standard: Torchcast Decision 4B: 62.1; attest-31B-v1.0: 69.7Standard62.1 · 69.7Judge: Torchcast Decision 4B: 43.4; attest-31B-v1.0: 67.4Judge43.4 · 67.4Hard: Torchcast Decision 4B: 49.8; attest-31B-v1.0: 63.9Hard49.8 · 63.9

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Per-tier competence on the sealed decisions, types pooled the same way; item text stays private.
How the categories were made

Chance-corrected competence (0 = chance, 100 = perfect) over the category's items the system saw, open and sealed pooled, tiers pooled: per request type the method's competence cell, then the mean over the request types present. Not part of the JevBench Score; never equated for API systems; compare systems within a category, not categories with each other. Values can be negative (below chance).

Family and language are authoring metadata of every item in the frozen v1.6 pool; where a uc1 item was written for a TypeSafe use case, that authoring use case is shown. This draw has no subject-topic labels yet, so the topic radar is omitted rather than filled from another draw. All non-English uc1 items are machine-authored and not native-reviewed.

  • Cells with fewer than 15 items are omitted; categories under 30 items are marked low-n.
  • Core v1.6 items carry no use case; the use-case view covers the use-case (uc1) items only.
All values as a table
SpokeA: Torchcast Decision 4BB: attest-31B-v1.0
The four score axes
Intelligence54.265.2
Calibration83.173.8
Speed93.697.1
Cost61.250.9
Use cases (TypeSafe categories)
Model routing69.982.4
Financial crime56.351.1
Competence per request type, open / sealed
Choice · open75.573.2
Choice · sealed62.874.6
Noul · open61.968.7
Noul · sealed50.668.2
Score · open42.052.9
Score · sealed32.553.8
Competence per tier — open set
Easy89.087.3
Standard65.355.9
Judge53.263.1
Hard59.767.3
Competence per tier — sealed set
Easy69.782.2
Standard62.169.7
Judge43.467.4
Hard49.863.9

Ten self-hosted rows and one vendor-hosted full-set API row on the full S1200+P300 draw; the full-set API row is not equated. Category cells use stored full-draw per-item results. The unranked A2 measurement carried from v1.6.14 has no category cells, and reflex-instinct-0.6b keeps its cells although it is not ranked. Sealed counts refer to self-hosted S (1,200); API rows use their original measured sealed basis.

Languages

Raw chance-corrected competence per item language (0 = chance, 100 = perfect; can be negative), from each system's own measured items: Self-hosted rows use S 1200 + P 300; API rows use their measured sealed subset plus P. Sage (A3) language cells and A2/A3 topic/use-case cells cover public P300 only. Raw and outside the Composite. Cells under 15 completed supported responses are left empty; recorded input refusals count toward coverage. A dagger (†) marks fewer than 30 completed responses. Competence retains all supported scored observations, including failures and refusals. Per-language coverage grows with the expanded uc1.1 multilingual pool, a candidate for a later release that is not part of v1.6.16. English (1,151 items) is listed first; the other 21 languages and the mixed-language group share 349 items. In 13 further groups no system reaches the 15-item reporting minimum, so they get no column (items in the pool shown): Mixed-language (14), French (14), Danish (13), Turkish (13), Swedish (11), Dutch (10), Korean (10), Chinese (10), Finnish (8), Czech (7), Ukrainian (7), Norwegian (6), Indonesian (4) — 127 items, scored like every other item.

10 systems

JevBench v1.6.16 competence by item language and system
1151393231282521161515
Torchcast Decision 4Bself-hosted · S+P5472616265†46†33†0†80†59†
attest-31B-v1.0API · S+P6758896774†45†82†26†53†67†
metask-jev-rain-12B-trainedself-hosted · S+P6572857481†45†59†25†49†77†
metask-jev-rain-31B-trainedself-hosted · S+P7163906968†46†73†36†57†78†
Uniphore-4B-Deciderself-hosted · S+P3225707152†29†0†18†74†15†
Standard One 3B SHself-hosted · S+P2820777270†22†20†0†68†21†
Krino-2Bself-hosted · S+P1643451221†22†0†0†19†12†
vLLM-SR d3self-hosted · S+P7579867379†39†51†69†73†55†
Phocinae-Largha-150M-v1self-hosted · S+P00000†0†0†0†0†0†
Julia-1self-hosted · S+P002203†0†0†0†0†1†

Intelligence gate and Noul decisiveness

The Composite applies a soft gate below Intelligence 50 (× (I/50)²). Rows under it show their value with a "below gate" label. The decisive rate is the share of valid Noul answers with P(yes) ≥ 0.80 or ≤ 0.20 (for systems that return only a label, every valid label counts as decisive); only decisive answers count as answers. Accuracy among decisive answers is a diagnostic and does not enter any score.

SystemIntelligenceChoiceNoulScoreNoul decisive rateAccuracy among decisive
Torchcast Decision 4B54.2nativenativenative100 %78 %
attest-31B-v1.065.2nativenativenative97 %88 %
metask-jev-rain-12B-trained67.8nativenativenative100 %86 %
metask-jev-rain-31B-trained70.7nativenativenative100 %88 %
Uniphore-4B-Decider34.2below gatenativenativenative72 %82 %
Standard One 3B SH32.8below gatenativenativenative63 %84 %
Krino-2B22.0below gatenativenativenative83 %71 %
vLLM-SR d373.7nativenativenative85 %96 %
Phocinae-Largha-150M-v13.6below gatenativenativenative3 %55 %
Julia-12.4below gatenativenativenative73 %46 %
Microsoft-Decision-1 (Azure Foundry) — A2 re-run59.2nativenativenativenot supported—
All data (12 systems)

Method notes · fresh native cohort · v1.6.16

Each system answered the same 1,200 freshly drawn sealed decisions and 300 public decisions on an offline GPU pod. Scores use methodology v1.6, O1S, with 1,000 bootstrap samples and seed 16.

Capability is the mean of Intelligence and Calibration within the frozen Jev-class cost and median-latency caps. Composite is secondary. Failed and refused answers remain in the full 1,500-decision denominator.

The public-versus-sealed gap reference is the median of this completed native cohort: 9.8 points. Historical scores are shown separately with their original dates and do not enter this cohort’s median or ranking. No API equating is applied.

Latency. API models are timed end to end from our measuring host, network included. Self-hosted models are timed on our own hardware over loopback (on-GPU, no network), so their measured value is not directly comparable with API latency. The ranking and the latency cap use the same estimate for every self-hosted row: 2 × measured p50 + 150 ms (est. incl. network (2x + 150 ms)). Tables show the measured value first, tagged “on-GPU, no network”, and the estimate second. A model author can provide a hosted endpoint; we then time it exactly like an API, network included, with no assumption (on request: info@productivity-boost.com, or note it in your submission).

Costs use each row’s documented price reference and measured usage on this draw. Subject-topic and use-case radars come from stored per-item results; only aggregates are published.

Sliders, presets and What-If change your view; they do not change the official result. Wrappers and subsidised systems are listed below the ranking.

Results SHA-256 8bb77523f5f430d2511c80934b465f26d163dc9d4870095be1c8520f78883bbe · categories SHA-256 ded9260b7368c9264e27ea443e4ca1a1c8ce91170ee9b120204beaf3f8de3d87 · scoring source SHA-256 d1939bdd3262d4ad65b2ade32c2cdccedb04589f3da4a51d5016ae29e5ae14fb.

Read these scores against this release only

Each JevBench release since v1.6 scores on its own fresh seeded draw, and the gap penalty is measured against the field median of that draw. This release’s median is 9.839526770355711, so its penalty threshold sits at 17.84; the live v1.6.1 board’s median is 2.573387642438244. That makes these numbers not directly comparable with the headline board or with earlier draws. It was frozen at v1.6.10 over its four members, so it is a small-field figure.

vLLM-SR d3 (vllm-sr-d3). Revision 5ce995dfb7ff07da00fec4211eb7f0560f52eaf8, base Qwen/Qwen3.8-27B, Apache-2.0 weights; the author does not disclose the training data. 1,500 of 1,500 items answered with 0 errors, and a 50-item determinism check was bit-identical. It was measured offline on one rented RTX PRO 6000 Blackwell 96 GB GPU because no single H100 was available, while this draw’s other GPU rows, Perplexity Decider and Torchcast-Decision-27B were timed on an H100, so its Speed axis rests on different hardware from those rows; the standard self-hosted ×2 + 0.15 s adjustment (an assumption) is applied on top. Its cost of USD 0.5463374 per 1,000 decisions is an estimate: the frozen Qwen3.8-27B list reference (the same basis as Perplexity Decider and Torchcast-Decision-27B) times its own measured tokens. That is far above the official Jev-class cost cap, so d3 is not on the default (capped) Capability ranking, and its composite is small because of the squared cost gate.

reflex-instinct-0.6b is shown but not ranked, exactly as in v1.6.15: CoS deputy decision #18761 accepts its Intelligence, Calibration and Cost and declares its Speed figure non-official, and the composite weights Speed. In the collapsed “All data” table further up this page, after the Noul table, it still shows the composite the scorer computed and a Jev-class verdict the site derives from its cost and median latency; neither is an official placement.

All eleven full-draw rows were scored in one invocation with O1S, 1,000 bootstrap samples, seed 16 and the frozen v1.6.10 field median, so every earlier row keeps its point score, axes, capability and cost. Ranks move: d3 joins at the top of this draw’s Capability order, so every ranked earlier row moves down one Capability place, and it joins between krino-2b and phocinae-largha-150m-v1 on the composites. No earlier row’s confidence interval moved; the notes below count every move. The carried A2 measurement is not rescored here and has no category cells.

Refusals, errors and context-bound failures stay in the full 1,500 denominator. Every row kept its own package, model, runtime and context settings; nothing was re-run for presentation.

NOT DIRECTLY COMPARABLE WITH THE LIVE v1.6.1 BOARD OR WITH THE EARLIER FRESH DRAWS. This release is a different draw with its own small-field median: G_med = 9.839526770355711, frozen from the four v1.6.10 members (attest-31B-v1.0 does not move it), against 2.573387642438244 on the live field. G_med_flag_gt10 is false. The gap penalty is max(0, 1 - max(0, (gap - G_med) - 8)/100), so this field's threshold is G_med + 8 = 17.84. On the merged live board this draw is placed with offset 0 while its anchor runs are pending, exactly like the earlier fresh draws. Measured, not estimated: Applying the same penalty formula to the stored base and gap at the live field's median 2.573387642438244, attest-31B-v1.0 Intelligence 65.238 would be 65.238 (gap -0.628); the original four rows are as stated in the v1.6.10 notes: standard_one_3b_sh Intelligence 32.786 would be 31.294 and metask_jev_rain_12b_trained 67.826 would be 67.587; krino-2b and metask_jev_rain_31b_trained are unchanged.

Cost: no item is excluded from the cost basis on this draw. Every row is priced from its own measured usage block at a clearly-labelled ESTIMATE rate fixed before scoring (r99, r101, r103 and r106 pre-registrations); none is a bill any of these systems issued. Every row is under the published v1.6 cost ceiling of USD 0.0646 per 1,000 decisions. The cost normalizer C0 is USD 0.001 per 1,000 decisions with 30 cost points per decade.

No subject-topic radar in this release: this draw's items have not been topic-labelled yet, and the shared topic labels of earlier draws do not cover its items. Families, use cases and languages are authoring metadata and are shown.

v1.6.13 ADDS ONE ROW, attest-31B-v1.0, to the six of v1.6.12 on the same draw, under the same rule as the earlier same-draw additions: the field median stays the frozen v1.6.10 value, so all six earlier rows keep identical point scores, axes and capability; ranks and bootstrap confidence intervals are recomputed for the seven-row roster, and the existing raw_on_A and API-basis offset diagnostics are unchanged because the new row is full-set API, not a new self-hosted equating-pool member. No API offset enters its score.

krino-2b: All 1,500 items answered HTTP 200 with native probabilities. The 15 longest items of the draw (the same 15 that standard_one_3b_sh refuses) ran with the state truncated to 65,536 tokens, as the author's submitted serve command documents (KEV_TRUNCATE_STATES=1); every other item is at most 1,802 input tokens. Reference kernels were used (KEV_FUSED=0) because the documented pip install has no flash-linear-attention. Kev reports usage.output_tokens as the token count of the serialized answer JSON although nothing is generated (165,276 over 1,500 decisions); it is priced as reported at the frozen DeepInfra Qwen/Qwen3.5-2B reference, USD 0.02/0.10 per 1M. Measured on an evaluator-owned Lium H100 pod in an offline container (network none).

metask_jev_rain_12b_trained: All 1,500 items answered HTTP 200. The author's service runs the authors' own int4 (W4A16) quantization of Google's Gemma 4 12B QAT checkpoint with their own rank-16 LoRA applied unmerged by vLLM; per its model card it takes a second pass on part of the items, so it reports generated output tokens (84,238 over 1,500 decisions) and its latency has a long tail (raw p95 8.1 s against a raw p50 of 0.13 s). Measured on an evaluator-owned Lium H100 80GB pod (IP geolocated to Nizhny Novgorod, Russia) in an offline container (network none). google/gemma-4-12b-it is not listed on OpenRouter, so the cost uses the nearest listed sibling, google/gemma-3-12b-it at USD 0.05/0.15 per 1M, times its own usage. It is a different system from the earlier metask_jev_rain_12b row.

metask_jev_rain_31b_trained: All 1,500 items answered HTTP 200. The author's service runs Google's unchanged W4A16 Gemma 4 31B QAT checkpoint with the authors' own rank-16 LoRA applied unmerged by vLLM, at max_model_len 6,656 (per the measuring operator's handoff): longer inputs are shortened by the service's own documented extraction rule rather than refused, so the longest items were answered on a shortened state. The authors tested it on H200; it was measured on an evaluator-owned Lium H100 80GB pod (IP geolocated to Nizhny Novgorod, Russia) in an offline container (network none) with unchanged settings. The service reports 1 output token per decision; cost is the OpenRouter google/gemma-4-31b-it list price (USD 0.09/0.34 per 1M) times its own usage.

standard_one_3b_sh: 15 of 1,500 items returned HTTP 422: they are the 15 longest items of the draw and exceed the model's documented 32,768-token context, so the author's server refuses them rather than truncating; scored wrong per method. Measured on an evaluator-owned Lium H100 pod in an offline container (network none). Priced at the M2 base-model floor, OpenRouter mistralai/ministral-3b-2512 at USD 0.10/0.10 per 1M, because the author's own list price (USD 0.038 per 1M input) is below that floor; the schema head generates no text, so output tokens are 0.

torchcast-decision-4b-v1.1: evaluator-owned Lium GPU pod (H100), offline read-only container (--network none); author's torchcast_decision server unchanged at tag v1.1.0 with the bundled decision_config.json (bf16, one forward pass reading option letters, calibration temperature 0.9, yes/no commit floor 0.801) on the documented stack torch 2.9.1+cu128 / transformers 5.8.1 / flash-linear-attention 0.5.2 / causal-conv1d 1.7.0 (fast kernels active, CUDA graphs on) Paid fast-lane order 529be5d5 (public). Qwen3.5-4B fine-tune served by the author's own torchcast_decision package. Runtime: official pytorch/pytorch:2.9.1-cuda12.8-cudnn9-devel image (digest-pinned) plus an offline hash-locked overlay of the documented packages (Python 3.11 instead of the documented 3.12). decision_config.json sets a disclosed yes/no commit floor of 0.801: a yes/no probability with max(P, 1-P) below 0.801 is moved to 0.801 on its own side (answer unchanged; same mechanism as h2o-lightning-4b). Prompts over 16,384 tokens keep the head and the tail of the state (documented). Output tokens are 0 (no generation). CC BY-NC 4.0 (weights); base Qwen/Qwen3.5-4B Apache-2.0; serving package Apache-2.0. ESTIMATE, v1.5-M2 base-model reference rule: self-hosted open weights, no bookable tariff. Base model Qwen/Qwen3.5-4B (model card base_model). OpenRouter does not list Qwen3.5-4B (host snapshot source/public-docs/OPENROUTER-MODELS.json, fetched 2026-10-12T08:15:26Z, read 2026-10-12). Binding precedent INTERPRETATION-1 I-2/I-3: DeepInfra Qwen/Qwen3.5-4B, USD 0.03/M input, USD 0.15/M output, frozen 25 Sep 2026 price cut-off snapshot (listing later deprecated; frozen reference kept). Logit readout, no generation: server reports 0 output tokens. Isolation is container-level (--network none); no separate host-level netcut probe was taken. The input tar was on the rented pod host disk during the run, and the pod was destroyed after measurement (CUSTODY-RECEIPT.json).

uniphore-4b-decider: 15 of 1,500 inputs exceed the author's documented 16,384-token wrapper limit; the author's code refuses them (HTTP 422, no truncation) and they are scored as wrong per method, so 1,485 items were answered. The native readout is one forward pass, so output tokens are 0 and the cost is the input tokens at the frozen DeepInfra Qwen/Qwen3.5-4B reference, USD 0.03/0.15 per 1M (that listing is flagged deprecated in the frozen snapshot but remains the reference). Measured on an evaluator-owned Lium H100 pod in an offline container (network none), in the vllm/vllm-openai v0.31.0 image with transformers 5.17 and reference PyTorch kernels in place of the documented flash-linear-attention / causal-conv1d kernels, so its latency was measured on the fallback path. The author discloses that the public JevBench set was inspected repeatedly during development.

attest-31b-v1.0: Paid public fast-lane order 07d13315. Vendor-hosted attest.engineering /v1/systemone API; latency measured from Sandy (Germany), using the official per-item latency subset without API adjustment. Full S1200+P300 draw, not API A300 supplement; no API equating. 15 of 1,500 requests returned HTTP 500 after 8 retry invocations and are scored as wrong in the full denominator. Public model documentation declares frozen google/gemma-4-31B-it @842da3794eaa0b77d5f08bae87a17459d91ff475, unmodified bf16, no adapter or quantization; single forward pass reading option-label logits, no generated tokens, one fitted temperature (value not documented). Hosting implementation and deployed weights were not independently inspected. Hosted API access is not an open-weights release by the vendor; base weights are under the Gemma 4 licence. ESTIMATE, v1.5-M2 base-model reference rule: vendor-hosted API, customer list price USD 0.048/M input and USD 0.00/M output effective 2026-10-11 has been in force fewer than 30 days. Base model google/gemma-4-31b-it reference on OpenRouter USD 0.09/M input / USD 0.34/M output, snapshot fetched 2026-10-12T17:55:05Z (SHA-256 81eae2f3f94bf299c8d4499260f842c357b7a869ce82dfa1741ad6a89cf57014), same base-model estimate precedent as metask_jev_rain_31b_trained. Applied to measured response usage; failed HTTP 500 usage/billing may be unreported. No output generation; reported output tokens 0. Estimated, not a vendor bill.

Microsoft A2 re-run: 300 sealed supplementary A2 + 300 public items, not the full S1200 set. Unranked measurement; the official Microsoft row is retained on the merged boards. Equating pool four of six: Krino, Standard One, Uniphore, MeTask 12B. MeTask 31B and Torchcast were not completed after four unsuccessful allocation attempts (price/creation/controller failures); no inferred scores for missing pool systems. Seven HTTP 422 responses are wrong in the 600-item denominator. Pool re-runs used H200 in Helsinki, versus H100 parents; wide equating confidence intervals are reported. Latency from Sandy (Germany). Cost is the Azure Foundry tariff USD 0.042/M input, USD 0/M output; invoice/warmup/refusal charges unknown. No category cells or ranks are assigned to the A2 measurement.

v1.6.15 ADDS THREE SELF-HOSTED CPU ROWS to the seven official rows of v1.6.14 on the same draw, under the same rule as the earlier same-draw additions: the field median stays the frozen v1.6.10 value, so all seven earlier rows keep identical point scores, axes, capability and cost objects. Ranks and bootstrap confidence intervals are recomputed over the new roster - the scorer draws ONE paired resample for the whole field, so adding rows moves a carried row's confidence interval while no point estimate changes; six of the eight carried rows' intervals moved (attest-31b-v1.0 and the carried A2 measurement did not). One DIAGNOSTIC also moved, and it is named here rather than left to be found: the API-basis median diagnostic is computed over the ranked field, so it moved from 9.628699345543296 to 9.158662835848954 when three rows joined, and that moves the stored A-basis diagnostic (the row field v16.raw_on_A) of exactly one carried row, standard_one_3b_sh: intelligence 29.3339 -> 29.1921, capability 53.1539 -> 53.0830. That diagnostic is not part of any score, rank or axis on this release, and no carried row's composite, axis, capability or cost object changes. The unranked A2 measurement of v1.6.14 is carried through byte for byte and is not rescored. reflex-instinct-0.6b is LISTED BUT NOT RANKED: CoS deputy decision #18761 accepts its Intelligence, Calibration and Cost only and declares its Speed figure non-official, and the composite weights Speed. All three new rows were measured at USD 0 on the evaluator's own shared CPU host, so their Speed figures are loaded-host lower bounds; each row logs its own one-minute load trace and carries it in its note.

Where to look for the row that is shown but not ranked, and what it does NOT mean. reflex-instinct-0.6b is deliberately absent from the ranking charts, the Composite and Capability boards, the compare view and the category radars, because those are ranked views and this row has no rank. It is named in this release's own notes and header, and it appears in the collapsed "All data" table further up the page, after the Noul table. DISCLOSED: in that table the row still shows its composite as the scorer computed it, and a Jev-class verdict the site derives from its cost per decision and its MEDIAN LATENCY against the frozen Jev-class caps. The composite weights the Speed axis, and that latency is the very measurement CoS deputy decision #18761 declares non-official for this row, so neither figure is an official placement. They are not an official placement and must not be quoted as one. Its accepted figures are Intelligence 3.960, Calibration 73.153, Capability 38.556 and Cost USD 0.00630127 per 1,000 decisions. A dedicated row group for listed-not-ranked rows, with Speed greyed out and the composite hidden, is the right presentation and is a site change this release does not make.

Cost on the three new rows: each is priced from its own measured usage block at a clearly labelled ESTIMATE rate fixed before scoring, on the full 1,500-row basis, and each is far under the published v1.6 ceiling of USD 0.0646 per 1,000 decisions - phocinae-largha-150m-v1 USD 0.00143980, reflex-instinct-0.6b USD 0.00630127, julia-1 USD 0.00217445. None is a bill any of these systems issued. julia-1's declared base jhu-clsp/mmBERT-small has no market listing; adding it to the frozen pricing module's reviewed unlisted-base set was authorised by #18761, re-frozen, hashed, proved behaviour-identical over every registry entry on the evaluator's machine and independently reviewed before this row was scored.

julia-1: 1,500 attempted, 1,483 answered. The 17 misses are the author's own strict_encoding policy refusing rather than truncating (15 items above their lossless context budget, 2 with an option above their 48-token option contract); the totals AND the per-type split were pre-registered before the run and came back exact. All answered items native; the Speed figures are the published method's percentiles over this draw's open standard and judge items. 652,334 input tokens and 0 output tokens by construction (one encoder forward pass read at marker positions, nothing generated). The author ships no server, so the HTTP surface is the evaluator's - but their own julia.typed.predict_typed already speaks JevBench's typed-question shape, so no mapping decision of the evaluator's enters this row. Their hierarchical Router, which their own card says is not a global probability distribution, is neither used nor reachable here: every choice item of this draw has 3 to 9 options, inside their native 2-20 contract, so each decision is one call with one global softmax, and the probabilities are their raw values rather than their presentation helper's. Measured on the evaluator's own shared CPU host at their own settings read back from the run itself (float32, 4 threads, torch backend); one-minute load 10.87-13.78 logged, so Speed is a loaded-shared-CPU lower bound with the standard self-hosted x2 + 0.15 s adjustment on top. Cost is a clearly labelled size-class ESTIMATE: the declared base jhu-clsp/mmBERT-small has no market listing, and the base-size-encoder line of USD 0.005 per 1M input / 0 output that modernbert-jev-v2, tacet-sonata and watt-flash-0.1 carry in the PUBLISHED v1.6 field is used unchanged - no row of this draw carries it - and it is not a bill.

phocinae-largha-150m-v1: 1,500 attempted, 1,125 answered. The 375 score items are recorded as HTTP 422 misses by the EVALUATOR'S shim, under a rule pre-committed in writing before the run (PREREG-R92 section 4.6), because the author's API answers only a fixed nine-level 2..10 scale that does not fit this item type; the shim refuses them in about 0.7 ms without calling the author's server at all, so no claim is made that the author's code refused anything. They stay in the full 1,500 denominator as misses - the figure 1,125 was pre-registered and came back exact. All answered items native. The Speed figures on this row are the published method's percentiles over this draw's open standard and judge items, not a median over every attempt - and DISCLOSED: 37 of those 124 items are the shim's instant score refusals, which cost no model time, so the figures are faster than the model itself is. Over the 87 items the model actually answered the Speed axis would read 75.75 instead of 77.12, which changes no rank. Measured on the evaluator's own shared CPU host (AMD Zen 2, float32), NOT a dedicated machine: other jobs ran beside it and the one-minute load average was logged for the whole window (3.75 to 9.89, mean 5.81, 12 samples - the quietest of this release's three new rows). Speed is therefore a loaded-shared-CPU figure and a lower bound for this model on a dedicated core or a GPU; the standard self-hosted x2 + 0.15 s adjustment is applied on top and the raw percentiles are kept. DISCLOSED: the author's server places every short-option request in its own 512-token budget tier, so the state is truncated far below the 8,192 its MAX_POS advertises - the author's design, not the evaluator's, and visible in their own usage histogram. A SECOND mapping decision of the evaluator's, footnoted on the row: the item's instruction text (and for noul its true/false criteria) is appended to the state, because state is the only text field the author's API has. Cost is a clearly labelled size-class ESTIMATE on the base-size-encoder line the PUBLISHED v1.6 field carries - modernbert-jev-v2 (measured in v1.6.8 on the earlier a2 draw) and the v1.6.1 rows tacet-sonata and watt-flash-0.1 - and no row of this draw carries it. It is not a bill.

reflex-instinct-0.6b: 1,500 attempted, 1,500 answered, zero failures of any class - the pre-registered figure, exact. All answered items native; the Speed figures are the published method's percentiles over this draw's open standard and judge items. NOT RANKED, and here is exactly why: CoS deputy decision #18761 accepts this row's Intelligence, Calibration and Cost only and declares its Speed figure (58.972) NON-OFFICIAL, because the one-minute load on the shared host ranged 9.42-33.12 (mean 15.28, 155 samples) during the window and the evaluator's own preparation activity overlapped it. The published composite weights Speed, so this row is shown with its axes and its Capability 38.556 - which is Intelligence and Calibration only and contains no Speed - but it carries no rank and no composite placement. Its Capability 38.556 is the highest of this release's three new rows, above phocinae-largha-150m-v1's 30.240 - said here because the Capability board does not show it: that board contains no Speed at all, so a reader could reasonably ask why this row is absent from it. The answer is that THIS RELEASE applies #18761 to the whole row rather than axis by axis - the deputy's decision excludes the Speed figure, and whether a row with a non-official Speed may still be ranked on Capability alone is the open question #18825, which has no answer yet. The conservative reading ships. DISCLOSED: the author's own encode() truncates state lines above its own max_len of 16,384 instead of refusing them, so this row answers the 15 items of 106,804-122,936 characters that four other rows of this draw recorded as context refusals (attest-31b-v1.0, julia-1, standard_one_3b_sh and uniphore-4b-decider); that is the author's design choice, not the evaluator's, and those items are the slowest in the row. Cost is a clearly labelled size-class ESTIMATE on the sub-1B line the PUBLISHED v1.6 field carries (liquid-d1-omni-600m, jiwo-0.8b, vegaml-0.8b) - no row of this draw carries it - and it is not a bill.

v1.6.16 ADDS ONE SELF-HOSTED GPU ROW, vllm-sr-d3 (vLLM-SR d3), to the ten full-draw rows of v1.6.15 on the same draw, under the same rule as the earlier same-draw additions: the field median stays the frozen v1.6.10 value, so every carried row keeps identical point scores, axes, capability, cost and speed objects. What moved on carried rows, counted exactly. RANKS: vllm-sr-d3 has the highest Capability of this draw, so all nine ranked carried rows move down one place in this release's Capability order (attest-31b-v1.0 3 -> 4, julia-1 9 -> 10, krino-2b 7 -> 8, metask_jev_rain_12b_trained 2 -> 3, metask_jev_rain_31b_trained 1 -> 2, phocinae-largha-150m-v1 8 -> 9, standard_one_3b_sh 6 -> 7, torchcast-decision-4b-v1.1 4 -> 5, uniphore-4b-decider 5 -> 6); its composite (A 5.570) sits between krino-2b and phocinae-largha-150m-v1, so phocinae-largha-150m-v1 (8 -> 9) and julia-1 (9 -> 10) also move down one place on composites A, B and C and in their headline rank; no other composite rank moves. CONFIDENCE INTERVALS: none of the ten carried rows' bootstrap confidence intervals moved. The scorer seeds each row's bootstrap with the shared seed 16 plus the row's position in the key-sorted roster, and vllm-sr-d3 sorts last, so no carried row's position changed (in v1.6.15 the three new keys sorted into the middle of the roster, which is why six carried intervals moved then). Correction to the v1.6.15 note above: it says the scorer draws ONE paired resample for the whole field; it does not - the scorer seeds each row's resample by its position (seed 16 + index in the key-sorted roster), each row with its own resample, which is why rows that changed position moved in v1.6.15 and none moved here. The same mechanism is why d3's interval here differs from its private single-row preview while its point values are identical: Capability 95% interval [78.928, 84.441] here, [79.337, 84.362] in the preview. DIAGNOSTICS: the API-basis median diagnostic is computed over the ranked field, so it moved from 9.158662835848954 to 9.40342052601045, which moves the stored A-basis diagnostic (the row field v16.raw_on_A) of exactly one carried row, standard_one_3b_sh: intelligence 29.1921 -> 29.2660, capability 53.0830 -> 53.1199; and the release-level equating-offset diagnostic moved from I 0.1329 / C 1.0866 to I 0.2288 / C 1.2340. Neither is part of any score, rank or axis on this release: the only API row scored in this invocation, attest-31b-v1.0, is a full-set row and is not equated, and the carried A2 measurement keeps its own A2 equating and is not rescored. reflex-instinct-0.6b stays LISTED BUT NOT RANKED under #18761 and the unranked A2 measurement of v1.6.14 is carried; neither is changed.

Cost on vllm-sr-d3, and the scope of the cost note above: that note's "every row is under the published v1.6 cost ceiling of USD 0.0646 per 1,000 decisions" is true of the ten rows of v1.6.15 and is NOT true of vllm-sr-d3. d3 is priced as an ESTIMATE at the frozen Qwen/Qwen3.8-27B list reference of the published v1.6 field (USD 0.42 per 1M input and USD 3.00 per 1M output, the same pair the published Perplexity Decider v1.1 27B and Torchcast-Decision-27B rows carry) times its own measured tokens on the full 1,500-row basis: 1,951,205 input tokens and 0 output tokens, USD 0.5463374 per 1,000 decisions, about 8.46x that ceiling. It is not a bill and not a quote from the author or any host. Consequences: its Cost axis is 17.876, below 50, so the squared cost gate of the composite applies and its composite A is only 5.570 (rank 8 of 10 here) despite the highest Capability of this draw; and on the merged live board it is far above the official Jev-class cost cap, so it is NOT on the default (cost-capped) Capability ranking.

Board effect, and why this release needed an explicit GO. On the merged live board, with the default caps, the top five of composites A, B and C and of the cost-capped Capability ranking are unchanged on the mixed board, on the open-weights board and on the API board (vllm-sr-d3 is outside the cost cap and its composite is far down). The views that change are Capability views with the cost cap switched off: with the latency cap at 2x and open-weights and API models mixed, vllm-sr-d3 becomes #1 on Capability, 82.966 against Perplexity Decider v1.1 27B at 82.792; on the open-weights board it is #1 with the cost cap off whether the latency cap is 2x or off; on the mixed board with both caps off it is #6 and that top five is unchanged. The #1 is not a resolved order: d3's 95% interval here is [78.93, 84.44] and Perplexity Decider's is [79.18, 84.71], they were measured on different draws (this draw sits on the merged board with offset 0 while its anchor runs are pending), and d3's Speed was timed on different hardware (see its note). A change to a public top five needs Florian's GO; he gave it on 13 Oct 2026 at 15:12Z (card 17392).

vllm-sr-d3: Model vllm-sr/d3 at revision 5ce995dfb7ff07da00fec4211eb7f0560f52eaf8 (vLLM Semantic Router team), a fine-tune of the base Qwen/Qwen3.8-27B; its model card states 26.09B parameters including a 0.46B vision encoder. Weights Apache-2.0. The training data is not disclosed by the author, so the weight licence alone says nothing about the provenance of the training data. 1,500 attempted, 1,500 answered (all HTTP 200), 0 errors, 0 missing; the author's server sets no input-length cap, so it also answered the draw's 15 longest items that several rows of this draw refuse. Determinism check: 50 items re-asked, 50/50 bit-identical probabilities, 0 argmax flips. HARDWARE DIFFERS FROM THIS DRAW'S OTHER GPU ROWS AND FROM PERPLEXITY DECIDER AND TORCHCAST-27B: measured offline on one rented NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB GPU (Verda), while this draw's other GPU rows, Perplexity Decider and Torchcast-Decision-27B were timed on a single H100 80 GB, because no single H100 was available; the Speed axis (87.369; p50 0.1087 s raw, 0.3675 s adjusted, over the scorer's 124 open standard and judge items) therefore rests on different hardware, and the larger 96 GB memory may also matter for the longest inputs. The plain CUDA/PyTorch path of the author's unmodified d3_server.py was measured; the author's AMD ROCm fast path does not apply on CUDA. The own-server adjustment x2 + 0.15 s is an assumption, not a measurement. Cost is an ESTIMATE, not a bill: the frozen Qwen3.8-27B list reference (the same basis as Perplexity Decider and Torchcast-Decision-27B) times d3's own measured tokens, USD 0.5463374 per 1,000 decisions, far above the official Jev-class cost cap, so d3 is NOT on the default (capped) Capability ranking and its composite is small because of the squared cost gate. Source review of the pinned code before the run: PASS (no network on the text path, no benchmark-specific branches, no secret reads).

Historical v1.3.0 board: weightings, per-task grid, topic radars and held-out diagnostics

Open this section to load the earlier public-only board and diagnostics.