A 10-second limit per decision; every row re-scored

What changed in v1.7.0

Method v1.7: from v1.7.0 a decision that takes longer than 10 seconds (the measured time, strictly more than 10.000 s) counts as a wrong answer and as zero calibration for that item (each item type's calibration is multiplied by the share of its decisions answered within 10 s). Every row is re-scored from its recorded per-decision times: Intelligence, Calibration, Capability, the composites and their intervals are recomputed with the same frozen scorer plus the rule; the Speed axis now uses all decisions given (it replaces the 124-item subset of v1.6). Cost, tokens, the draws, the raw outputs and the field median are unchanged; no model was run again for the re-scored rows. Every re-scored row shows its v1.6 values next to the new ones.

Time limit on measured time: the limit uses the measured latency, not the displayed estimate (x2 + 0.15 s for self-hosted rows), so a self-hosted row can show an adjusted p95 above 10 s with no decision over the limit. In total 1,455 decisions across 59 rows are over the limit.

Re-measure pending: 5 rows (chon-1.0-e4b, nev-2b, modernbert-jev-v2, mghafiri-qwen3.5-0.8b-decision-model, reflex-instinct-0.6b) recorded their per-decision times on a loaded CPU host, so they cannot be scored under the time limit and no estimate is published. They stay on the board with their v1.6 values, labelled as v1.6, unranked, until they are measured again.

Re-measured in v1.7.0: 4 rows that were pending (loaded CPU host) were measured again on the same items on a dedicated GPU. Each GPU row replaces the earlier CPU row in full, scores and times; GPU times are never combined with CPU scores, and the CPU values stay visible as the row's v1.6 values. pjev-2b (Kailune AI, frozen MiniCPM5-2B + merged LoRA r64, letter-logit readout, T=1 stub) (pjev-2b): no decision took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw. PurpleMIST-Mini-1.0 (Olaverse) (purplemist-mini-1.0): no decision took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw. Vega-U (Qwen3.5-4B reader) (vega-u-qwen3.5-4b): 15 of its 1,500 decisions took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw. Verdin-4B (Victor Bennett, Qwen3.5-4B + merged LoRA r16 + pointer head) (verdin-4b): no decision took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw.

Open/sealed gap penalty: the public-versus-sealed gap penalty is computed from the row's own results, so a decision that now fails can slightly reduce that penalty and raise Intelligence a little. This affects 1 row in this release (shisa-de-1 (shisa-ai, Gemma-4-26B-A4B): +0.006 Intelligence points); it is a side effect of the existing penalty, not a change to it.

Dated carry: 14 rows measured before v1.6 (v1.5 method) are carried as published and not re-scored.

Equated API rows: the pool re-runs behind the A4 and A5 equating offsets are scored under the same rule, so the offsets moved with them. These rows keep the 2-decimal precision of their earlier publication.

New in v1.7.0: 26 rows measured for the first time under the 10 s rule. AJev lm6q (Gemma 4 31B) (ajev-gemma4-31b-lm6q): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Aleria Judge 1.1.1 (ALERIA-ML, inference technique on stock Gemma 4 31B, author server, self-hosted) (aleria-judge-1.1.1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Arcbright Elf 1 31B (arcbright-elf-1-31b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Bobcat 1.3 (Gemma 4 31B) (bobcat-1.3): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. DecisionNode Edge 0.9.2 (decisionnode-edge-0.9.2): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. DecisionNode Edge Mini 0.9.2 (decisionnode-edge-mini-0.9.2): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. baselabs elephant-1 (9B) (elephant-1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Fastino GLiDE 1.2 (open weights, self-hosted) (fastino-glide-1.2-open): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Jeffan-31b-1710 (jeffan-31b-1710): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Jeffan-4B-2010 (jeffan-4b-2010): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Jez-large-1.0.0 (pragmaticfrontier, Gemma-4-E4B label-logit decision model, author server, self-hosted) (jez-large-1.0.0): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Jez-small-1.0.0 (pragmaticfrontier, Qwen3.5-0.8B label-logit decision model, author server, self-hosted) (jez-small-1.0.0): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. KnowLine 31B-Gen1 (knowline-31b-gen1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. KnowLine 35B-A3B-Gen1.5 (knowline-35b-a3b-gen1.5): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. FrontiersMind Lumma-fev-27B (lumma-fev-27b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. baselabs mammoth-1 (27B) (mammoth-1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Nimble-v4-max (open weights, self-hosted) (nimble-v4-max-open): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 1 of its 1,500 decision took longer than 10 seconds and is scored by the rule. Its mandatory disclosure and licence note are shown on the row. Seb-9B-v2 (ironbcc, Qwen3.5-9B fine-tune, one forward pass, stock vLLM, self-hosted) (seb-9b-v2): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Super Decider (singularity-super-decider): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 11 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Super Decider (thinking) (singularity-super-decider-thinking): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 4 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Standard One 3B SH v4 (Standard Thinking) (standard_one_3b_sh_v4): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Tern-35B-A3B (matthewagi, Qwen3.6 35B-A3B MoE decision model, author server, self-hosted) (tern-35b-a3b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Vera Large (vera-large): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. vLLM-SR d3-mini (vllm-sr-d3-mini): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Wald 26B-A4B (Gemma 4 26B-A4B) (wald-26b-a4b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Xevn-4B (xevn-4b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row.

Board effect: the 10 s rule (method v1.7) re-scored every row; the merged-board top fives before/after v1.7.0 were compared over all 36 mixed, open-weights and API variants (including cost caps of 7x and 10x the reference cost, each also with no latency cap). The reference cost is that of Jev 1.13.0, USD 0.0323 per 1,000 decisions, and the official cap is 2x that. Together with 4 release-board lists, 37 of 40 top-five lists change. Top-five changes caused by: the 10 s rule (method v1.7) and the new rows arcbright-elf-1-31b, fastino-glide-1.2-open, knowline-31b-gen1, mammoth-1, singularity-super-decider, singularity-super-decider-thinking (measured for the first time in v1.7.0); entering a top five: arcbright-elf-1-31b, fastino-glide-1.2-open, kettei-fat, knowline-31b-gen1, mammoth-1, singularity-super-decider, singularity-super-decider-thinking; leaving a top five: chon-1.0-12b, drex-max, fastino-glide-api, h2o-lightning-31b, liquid-d1-ultra, nimble-v4-max, qwen3.8-27b, subq-snap-1.1, subq-snap-preview, vllm-sr-d3, vllm-sr-d3-turbo.

Publication of v1.7.0: Florian selected "Publish (recommended)" on Telegram card #18506 (2026-10-24T14:13:05Z), the top-five preview card with preview PNG sha256 b082a945643c70be13f1628cfb9a908f4d11f1acdc72e33b2785ffecad617d2e. Florian top-five preview approval: Telegram card #18506; selected "Publish (recommended)" (2026-10-24T14:13:05Z); preview PNG SHA-256 b082a945643c70be13f1628cfb9a908f4d11f1acdc72e33b2785ffecad617d2e.