Benchmarks
More
OverviewBenchmaxxingCompareChartsCost vs CapabilityProviders per ModelProvider explorerGatewaysEU & SovereignJev-class modelsImage JevBenchAudioJevBenchSubmit a modelAbout
JevBench v1.7.2 · same-draw addition

One decision model added

Jeetu Decider 2B joins on the same draw. The 10-second rule and common cost basis are unchanged. All carried scores and confidence intervals are retained; only placements move.

Aggregate results JSON · SHA-256 754f776d1a14660936eea464070991243b0049aebda140ea5ff25c590e4fe2a7 · Previous v1.7.1

JevBench v1.7.2 · headline

JevBench Capability Score

Capability ranking of Jev-class systems

Capability Score averages Intelligence and Calibration. kettei-fat leads the Jev-class systems with 86.3.

Jev-class means at most 2× the cost and median latency of Jev (v1.5 reference). How we choose ↘

Adjust cost / latency caps · 2× official
Official
#SystemICCap.$/1k
  1. 1kettei-fatAPIv1.6.43 drawv1.7 · 10 s rule79.857.486.3$0.026*
  2. 2Super DeciderAPIv1.7.0 drawv1.7 · 10 s rule80.056.485.9$0.028
  3. 3Super DeciderAPIv1.7.0 drawv1.7 · 10 s rule78.964.384.9$0.016
  4. 4Fastino GLiDEAPIv1.6.43 drawv1.7 · 10 s rule76.159.184.5$0.023
  5. 5H2O-Lightning-31B v1.0.7v1.6.43 drawv1.7 · 10 s rule79.350.684.3$0.044*
  6. 6Drex MaxAPIv1.6.43 drawv1.7 · 10 s rule78.955.384.2$0.031
  7. 7basal-omni-1.0-ultrav1.6.43 drawv1.7 · 10 s rule74.150.482.1$0.045*
  8. 8SubQ Snap 1.1APIv1.6.43 drawv1.7 · 10 s rule75.860.781.9$0.020*
  9. 9Arcbright Elf 1 31Bv1.7.0 drawv1.7 · 10 s rule71.949.781.8$0.047*
  10. 10Quyet-1.0-Largev1.7 · 10 s rule73.450.581.7$0.045*
Show all 193 Jev-class systems (183 more)
  1. 11Jeff-1.0-Largev1.6.4 drawv1.7 · 10 s rule71.949.281.5$0.049*
  2. 12SubQ SnapAPIv1.6.43 drawv1.7 · 10 s rule72.357.781.5$0.026*
  3. 13d1-ultraAPIv1.6.43 drawv1.7 · 10 s rule69.762.380.5$0.018*
  4. 14Bespoke Nimble-v4-maxAPIv1.6.43 drawv1.7 · 10 s rule68.260.779.7$0.020*
  5. 15metask-jev-rain-31B-trained service v3v1.6.43 drawv1.7 · 10 s rule72.649.379.6$0.049*
  6. 16decisio v0.8.0 on gemma-4-31B-itv1.7 · 10 s rule70.451.779.6$0.041*
  7. 17metask-jev-rain-31B-trainedv1.6.43 drawv1.7 · 10 s rule70.750.778.9$0.044*
  8. 18Myles-G 31BAPIv1.6.43 drawv1.7 · 10 s rule73.651.178.8$0.043*
  9. 19Sage 1.3.0APIv1.7 · 10 s rule65.658.278.6$0.025*
  10. 20Bobcat 1.3v1.7.0 drawv1.7 · 10 s rule68.149.977.9$0.047*
  11. 21deck-31Bv1.7 · 10 s rule73.049.777.6$0.048*
  12. 22Torchcast Decision 4Bv1.6.43 drawv1.7 · 10 s rule67.164.877.5$0.015*
  13. 23Jev 1.13.0APIv1.7 · 10 s rule63.654.777.1$0.032
  14. 24metask-jev-rain-12B-trainedv1.6.43 drawv1.7 · 10 s rule66.552.776.9$0.038*
  15. 25Wald 26B-A4Bv1.7.0 drawv1.7 · 10 s rule65.549.676.8$0.048*
  16. 26René-1 31B FP8v1.7 · 10 s rule61.746.076.0$0.063*
  17. 27Jev 1.14.0APIv1.6.43 drawv1.7 · 10 s rule59.954.875.0$0.032*
  18. 28H2O-Lightning-4B v1.1v1.6.43 drawv1.7 · 10 s rule60.064.075.0$0.016*
  19. 29d1APIv1.7 · 10 s rule62.162.874.4$0.017
  20. 30SPX-CD FlashAPIv1.7 · 10 s rule58.452.174.3$0.039*
  21. 31Mercury DecideAPIv1.7 · 10 s rule65.962.173.6$0.018*
  22. 32Celeris-1 DecisionAPIv1.6.43 drawv1.7 · 10 s rule58.960.473.6$0.021*
  23. 33OpenAI DecisionsAPIv1.7 · 10 s rule56.948.673.5$0.052
  24. 34Surogate Rune 26B-A4B v3v1.7 · 10 s rule56.149.673.5$0.048*
  25. 35kettei-chubAPIv1.6.43 drawv1.7 · 10 s rule62.262.773.1$0.018*
  26. 36Bobcat Flash 1.2v1.7 · 10 s rule65.649.773.0$0.048*
  27. 37Torchcast Decision 4Bv1.6.43 drawv1.7 · 10 s rule60.364.872.6$0.015*
  28. –Microsoft-Decision-1APIv1.6.43 drawv1.7 · 10 s rule59.262.272.0$0.018PRELIMINARY — API supplement A2 (300 sealed + 300 public), not the S1200 full set; equating pool 4 of 6
  29. 38torchcast-decision-12bv1.7 · 10 s rule60.556.471.7$0.028*
  30. 39decider-12b v2v1.7 · 10 s rule61.957.871.3$0.026*
  31. 40Jev-Omniv1.7 · 10 s rule55.556.171.3$0.029*
  32. 41Winnow-12B Q8v1.7 · 10 s rule59.556.671.2$0.028*
  33. 42Xor 26B-A4Bv1.7 · 10 s rule57.550.771.2$0.044*
  34. 43decider-12b v1, stock Gemma-4-12B-itv1.7 · 10 s rule59.657.871.0$0.026*
  35. 44Cygnetv1.7 · 10 s rule54.856.470.9$0.028*
  36. 45Microsoft-Decision-1APIv1.7 · 10 s rule57.261.570.8$0.019
  37. 46decisio v0.8.0 on gemma-4-12B-itv1.7 · 10 s rule52.359.370.7$0.023*
  38. 47Myles-G 12BAPIv1.6.43 drawv1.7 · 10 s rule57.357.470.7$0.026*
  39. 48InstinctAPIv1.7 · 10 s rule47.864.470.5$0.015
  40. 49decider chat on Gemma-4-31B-itv1.7 · 10 s rule58.550.870.4$0.044*
  41. 50baselabs elephant-1v1.7.0 drawv1.7 · 10 s rule55.848.470.2$0.052*
  42. 51GEV-26B-Decidev1.7 · 10 s rule49.851.170.1$0.043*
  43. 52Chon-1.0-12B-Fastv1.7.1 drawv1.7 · 10 s rule60.358.770.1$0.024*
  44. 53decisio v0.9.0 on gemma-4-12B-itv1.7 · 10 s rule52.359.370.0$0.023*
  45. 54Blink v0.3 26B-A4Bv1.6.9 drawv1.7 · 10 s rule49.249.669.9$0.048*
  46. 55attest-31B-v1.0APIv1.6.43 drawv1.7 · 10 s rule65.250.769.5$0.044*
  47. 56Wald 4B v2.1v1.6.9 drawv1.7 · 10 s rule52.046.369.3$0.062*
  48. 57vLLM-SR d3-miniv1.7.0 drawv1.7 · 10 s rule50.864.268.8$0.016*
  49. 58Torchcast Decision 4Bv1.6.43 drawv1.7 · 10 s rule54.264.868.7$0.015*
  50. 59Xevn-4Bv1.7.0 drawv1.7 · 10 s rule52.166.068.6$0.014*
  51. 60Hopper 12B trainedv1.7 · 10 s rule51.756.268.1$0.029*
  52. 61SPX-CD-Omniv1.7 · 10 s rule48.858.368.1$0.025*
  53. 62Aplomb 1v1.7 · 10 s rule45.257.967.1$0.025*
  54. 63Bespoke Nimble 9B v3v1.7 · 10 s rule56.546.166.8$0.063*
  55. 64Amazon Strands Decider 26B-A4Bv1.7.1 drawv1.7 · 10 s rule46.750.566.7$0.045*
  56. 65Metask rain 4Bv1.6.4 drawv1.7 · 10 s rule51.151.866.0$0.040*
  57. 66shisa-de-1v1.6.43 drawv1.7 · 10 s rule49.949.865.5$0.047*
  58. 67Plumb-4Bv1.7 · 10 s rule42.863.165.3$0.017*
  59. 68Arcbright Elf 1 12Bv1.6.43 drawv1.7 · 10 s rule43.857.465.0$0.026*
  60. 69decider-4b v2v1.7 · 10 s rule40.164.564.9$0.015*
  61. 70Seb-9B-v2v1.7.0 drawv1.7 · 10 s rule46.148.964.8$0.051*
  62. 71HS1 0.1.2APIv1.6.43 drawv1.7 · 10 s rule52.949.164.3$0.050*
  63. 72JevK5 v0.3v1.7 · 10 s rule38.563.164.2$0.017*
  64. 73Bespoke Nimble-v4APIv1.6.43 drawv1.7 · 10 s rule41.460.764.0$0.020*
  65. 74Jeffan-4B-2010v1.7.0 drawv1.7 · 10 s rule40.164.164.0$0.016*
  66. 75Clef-Flashv1.7 · 10 s rule42.246.863.8$0.059*
  67. 76JPT-4Bv1.6.43 drawv1.7 · 10 s rule40.864.263.8$0.016*
  68. 77Vansa-3.4APIv1.7 · 10 s rule40.961.963.7$0.019
  69. 78deck-4B v1.0v1.7 · 10 s rule38.663.563.6$0.017*
  70. 79Jebadiah 9B v2.1v1.6.43 drawv1.7 · 10 s rule37.248.963.4$0.051*
  71. 80Seb-9Bv1.6.43 drawv1.7 · 10 s rule38.348.963.1$0.051*
  72. 81janus 4Bv1.7 · 10 s rule37.664.462.9$0.015*
  73. 82Amos 1 mediumAPIv1.6.43 drawv1.7 · 10 s rule38.148.762.9$0.051*
  74. 83JevK5 v0.2.0v1.7 · 10 s rule36.363.162.8$0.017*
  75. 84Messier One v0.4v1.6.43 drawv1.7 · 10 s rule52.061.162.4$0.020*
  76. 85Imajev-4Bv1.7 · 10 s rule34.663.361.9$0.017*
  77. 86jqvv1.7 · 10 s rule33.951.261.9$0.042*
  78. 87metask-jev-4bv1.7 · 10 s rule39.257.761.8$0.026*
  79. 88Kahn1 4Bv1.6.43 drawv1.7 · 10 s rule39.750.161.7$0.046*
  80. 89pplx-decider-flash-v1.2v1.6.43 drawv1.7 · 10 s rule45.564.461.7$0.015*
  81. 90levv1.7 · 10 s rule41.161.861.7$0.019*
  82. 91Decision 2.0 Lux 9Bv1.6.43 drawv1.7 · 10 s rule35.848.561.6$0.052*
  83. 92blink-4bv1.6.43 drawv1.7 · 10 s rule35.063.561.4$0.016*
  84. 93Hopperv1.7 · 10 s rule33.662.361.2$0.018*
  85. 94Malkuth-4Bv1.7 · 10 s rule33.855.861.1$0.030*
  86. 95Decision 4B v1.2v1.7 · 10 s rule32.463.160.4$0.017*
  87. 96jiwo 4bv1.6.9 drawv1.7 · 10 s rule34.864.160.2$0.016*
  88. 97Verdin-4Bv1.7.0 drawv1.7 · 10 s rule35.462.259.6$0.018*
  89. 98Wald 4Bv1.6.4 drawv1.7 · 10 s rule41.054.359.6$0.033*
  90. 99Quyet-1.0-Mediumv1.7 · 10 s rule41.763.459.2$0.017*
  91. 100Decision-4Bv1.7 · 10 s rule37.965.959.1$0.014*
  92. 101Manchego v2.1v1.7 · 10 s rule32.564.358.3$0.015*
  93. 102Messier One v0.2v1.7 · 10 s rule42.464.858.1$0.015
  94. 103Diffusion Jevv1.7 · 10 s rule54.949.658.1$0.048*
  95. 104jev-localv1.7 · 10 s rule41.958.957.5$0.024*
  96. 105Instinct Dual 4BAPIv1.7 · 10 s rule28.375.057.2$0.0068
  97. 106Decision 4B v1.1v1.7 · 10 s rule29.163.157.2$0.017*
  98. 107JEV Qwen3.5-9B Base NVFP4v1.7 · 10 s rule29.347.657.1$0.056*
  99. 108iota-decide-4bv1.6.43 drawv1.7 · 10 s rule29.463.057.1$0.017*
  100. 109Uniphore-4B-Deciderv1.6.43 drawv1.7 · 10 s rule34.264.557.1$0.015*
  101. 110Jebadiah 9B v2v1.6.43 drawv1.7 · 10 s rule30.648.956.9$0.051*
  102. 111CoCo-Decision-4Bv1.7 · 10 s rule32.765.256.9$0.014*
  103. 112APUS-OpenJev-v1-9Bv1.7 · 10 s rule47.248.756.9$0.051*
  104. 113Decider-4Bv1.6.43 drawv1.7 · 10 s rule27.166.256.8$0.013*
  105. 114typecastlmv1.7 · 10 s rule29.864.056.5$0.016*
  106. 115pjev-2bv1.7.0 drawv1.7 · 10 s rule28.863.556.5$0.016*
  107. 116TypeCastLM 1.4.0v1.7 · 10 s rule35.464.856.4$0.015*
  108. 117RYOTIDE-Qwenv1.6.43 drawv1.7 · 10 s rule25.262.256.3$0.018*
  109. 118Compass 0.2.0v1.6.43 drawv1.7 · 10 s rule28.957.656.1$0.026*
  110. 119Standard One 3B SH v4v1.7.0 drawv1.7 · 10 s rule34.448.656.0$0.052*
  111. 120Standard One 3B SHv1.6.43 drawv1.7 · 10 s rule32.848.655.6$0.052*
  112. 121SemIfv1.7 · 10 s rule27.163.155.4$0.017*
  113. 122ArseneLupin v1.1v1.6.43 drawv1.7 · 10 s rule28.060.755.1$0.020*
  114. 123local-jev Qwen3.5-4Bv1.7 · 10 s rule24.159.355.0$0.023*
  115. 124Decision 2Bv1.7 · 10 s rule22.966.454.8$0.013*
  116. 125Jez-large-1.0.0v1.7.0 drawv1.7 · 10 s rule25.670.554.4$0.0096*
  117. 126Nomav1.6.43 drawv1.7 · 10 s rule25.162.054.3$0.019*
  118. 127spark-s1-4b-v6v1.7 · 10 s rule43.260.454.0$0.021*
  119. 128Imajev-2Bv1.6.43 drawv1.7 · 10 s rule21.251.854.0$0.041*
  120. 129RYOTIDE-Qwen9v1.6.4 drawv1.7 · 10 s rule27.649.452.5$0.049*
  121. 130Arbiter-4Bv1.6.43 drawv1.7 · 10 s rule27.466.252.3$0.013*
  122. 131DecisionNode Edge 0.9.2v1.7.0 drawv1.7 · 10 s rule26.871.252.1$0.0091*
  123. 132system-one-openAPIv1.7 · 10 s rule27.968.351.9$0.011*
  124. 133ClassOne Qwen 3.5 9Bv1.7 · 10 s rule42.249.350.5$0.049*
  125. 134Liquid AI d1-3Bv1.6.9 drawv1.7 · 10 s rule22.150.649.5$0.044*
  126. 135ZeroEntropy zerank-2v1.7 · 10 s rule15.648.449.4$0.052*
  127. 136open-alternative-jevv1.7 · 10 s rule22.363.349.1$0.017*
  128. 137OpenSourceJevv1.7 · 10 s rule23.269.148.6$0.011*
  129. 138EXAONE-4.0-1.2B-JEV v0.3v1.7 · 10 s rule14.857.947.7$0.025*
  130. 139Malkuth-2Bv1.7 · 10 s rule15.965.647.6$0.014*
  131. 140DecisionNode Edge Mini 0.9.2v1.7.0 drawv1.7 · 10 s rule17.680.247.6$0.0046*
  132. 141Nemotron Diffusion 8Bv1.7 · 10 s rule20.455.547.5$0.030*
  133. 142jiwo 0.8bv1.6.9 drawv1.7 · 10 s rule15.978.447.2$0.0052*
  134. 143decision-machine-1APIv1.7 · 10 s rule13.158.346.1$0.024
  135. 144decisor-4bv1.6.4 drawv1.7 · 10 s rule39.664.045.7$0.016*
  136. 145Qwen3-Reranker-4Bv1.7 · 10 s rule16.348.444.7$0.052*
  137. 146decider-2bv1.7 · 10 s rule20.764.944.6$0.015*
  138. 147BAAI bge-reranker-v2-m3v1.7 · 10 s rule0.159.344.1$0.023*
  139. 148kev 4Bv1.7 · 10 s rule19.565.844.0$0.014*
  140. 149Mixedbread mxbai-rerank-base-v2v1.7 · 10 s rule1.160.343.9$0.021*
  141. 150lev-350mv1.7 · 10 s rule4.880.043.3$0.0046*
  142. 151Jez-small-1.0.0v1.7.0 drawv1.7 · 10 s rule4.679.743.2$0.0048*
  143. 152Certo v1v1.7 · 10 s rule0.296.943.2$0.0013*
  144. 153PurpleMIST-Mini-1.0v1.7.0 drawv1.7 · 10 s rule8.470.843.0$0.0094*
  145. 154Chon-1.0-12Bv1.6.43 drawv1.7 · 10 s rule27.349.043.0$0.050*
  146. 155smalljev semantic-v9v1.7 · 10 s rule8.760.843.0$0.020*
  147. 156Vega-Uv1.7.0 drawv1.7 · 10 s rule8.564.942.6$0.015*
  148. 157jul fastv1.7 · 10 s rule7.569.042.6$0.011*
  149. 158Krino-2Bv1.6.43 drawv1.7 · 10 s rule22.061.042.4$0.020*
  150. 159Alibaba GTE Reranker ModernBERT-basev1.7 · 10 s rule0.069.341.7$0.011*
  151. 160Jeetu Decider 2Bv1.7.2 drawv1.7 · 10 s rule10.270.341.7$0.0098*
  152. 161watt-flash-0.1v1.7 · 10 s rule3.889.041.6$0.0023
  153. 162Decision Fastv1.7 · 10 s rule6.180.140.6$0.0046*
  154. 163openJev Verdict 1.4v1.7 · 10 s rule5.086.640.4$0.0028*
  155. 164Quyet-1.0-Small-ENv1.7 · 10 s rule6.189.139.5$0.0023*
  156. 165ClassOne Gemma 4 E4Bv1.6.9 drawv1.7 · 10 s rule28.269.338.4$0.011*
  157. 166ClassOne Qwen 3.5 4Bv1.6.9 drawv1.7 · 10 s rule27.365.038.1$0.015*
  158. 167OpenDecisionv1.7 · 10 s rule7.479.138.0$0.0050*
  159. 168verdict-smallv1.7 · 10 s rule2.3100.037.6$0.00087*
  160. 169kev 0.6Bv1.7 · 10 s rule7.680.136.9$0.0046*
  161. 170Vega 4Bv1.6.9 drawv1.7 · 10 s rule15.747.236.6$0.058*
  162. 171WaterSheepv1.7 · 10 s rule5.368.735.5$0.011*
  163. 172Raw Phi-4 mini direct logitsv1.7 · 10 s rule13.753.235.3$0.036*
  164. 173Fastino GLiNER-2.5-DecideAPIv1.7 · 10 s rule7.045.834.6$0.064
  165. 174Raw Qwen3 4B Instruct 2507 direct logitsv1.7 · 10 s rule34.063.434.6$0.017*
  166. 175Vega 0.8Bv1.6.9 drawv1.7 · 10 s rule3.161.533.0$0.019*
  167. 176kev 0.5Bv1.7 · 10 s rule4.680.132.9$0.0046*
  168. 177Liquid AI d1-omni-600Mv1.6.9 drawv1.7 · 10 s rule3.373.132.8$0.0079*
  169. 178Quyet-1.0-Smallv1.7 · 10 s rule4.079.632.5$0.0048*
  170. 179Quyet-1.0-Tinyv1.7 · 10 s rule3.788.431.1$0.0024*
  171. 180Open Jev JSON Canvasv1.7 · 10 s rule61.249.330.6$0.049*
  172. 181Phocinae-Largha-150M-v1v1.6.43 drawv1.7 · 10 s rule3.695.330.2$0.0014*
  173. 182JevActAPIv1.7 · 10 s rule10.168.628.8$0.011*
  174. 183openJev Verdictv1.7 · 10 s rule4.186.626.6$0.0028*
  175. 184Tacet Sonatav1.7 · 10 s rule2.289.024.2$0.0023*
  176. 185CLM-8Bv1.7 · 10 s rule0.150.520.1$0.045*
  177. 186ClassOne Qwen 3.5 2Bv1.6.9 drawv1.7 · 10 s rule5.870.219.2$0.0098*
  178. 187Deem 0.8B v1v1.7 · 10 s rule6.580.318.1$0.0045*
  179. 188Laya multilingualv1.7 · 10 s rule0.382.217.0$0.0039*
  180. 189ClassOne Gemma 4 E2Bv1.7 · 10 s rule7.870.215.2$0.0098*
  181. 190Julia-1v1.6.43 drawv1.7 · 10 s rule2.489.79.9$0.0022*
  182. 191Raw Qwen3 1.7B direct logitsv1.7 · 10 s rule5.168.58.3$0.011*
  183. 192Raw Qwen3 0.6B direct logitsv1.7 · 10 s rule5.677.68.2$0.0056*

Wide coloured bar = Capability Score (0–100). Thin bars: cost above, median latency below; shared log ratio scale 0.25× … 64× Jev (v1.5 reference), ticks at 1× and 2× (cap). Shorter is cheaper or faster. * = est. (estimated cost). # counts ranked Jev-class systems; “–” marks unranked or outside systems. Tap ⓘ or hover a row or thin bar for details.

Cost and latency correlate here (Spearman ρ = 0.27, n = 274).

  • Jev — reference (TypeSafe, closed) (2)
  • Closed API (weights not public) (31)
  • Open weights · LLM decoder (201)
  • Open weights · diffusion LM (6)
  • Open weights · encoder / classifier (21)
  • Open weights · reranker (5)
  • Base model control (no decision fine-tune, raw logits) (5)
  • System (router / cascade / ensemble) (3)
  • green: ≤ reference
  • amber: ≤ cap (2× reference)
  • red: > cap
Show general-purpose LLMs and other systems outside the limits

Sorted by Capability Score, not numbered. Each row says which limit it misses, measured against Jev ($0.032 per 1,000 decisions, median 0.62 s).

  1. –GPT-6 Luna (medium)APIv1.7 · 10 s rule96.938.498.4$0.11Outside: cost 3.5× Jev (v1.5 reference), latency 3.1× Jev (v1.5 reference)
  2. –GPT-6 Luna (low)APIv1.7 · 10 s rule95.739.097.1$0.11Outside: cost 3.3× Jev (v1.5 reference), latency 3.1× Jev (v1.5 reference)
  3. –GPT-5.6 LunaAPIv1.7 · 10 s rule93.030.293.6$0.21Outside: cost 6.6× Jev (v1.5 reference), latency 2.6× Jev (v1.5 reference)
  4. –DeepSeek V4.1 FlashAPIv1.7 · 10 s rule84.519.689.6$0.48Outside: cost 14.9× Jev (v1.5 reference), latency 3.6× Jev (v1.5 reference)
  5. –baselabs mammoth-1v1.7.0 drawv1.7 · 10 s rule76.239.585.2$0.10*Outside: cost 3.2× Jev (v1.5 reference)
  6. –KnowLine 31B-Gen1v1.7.0 drawv1.7 · 10 s rule77.440.985.2$0.093*Outside: cost 2.9× Jev (v1.5 reference)
  7. –Fastino GLiDE 1.2v1.7.0 drawv1.7 · 10 s rule76.430.684.9$0.21*Outside: cost 6.4× Jev (v1.5 reference)
  8. –Perplexity Decider v1.1 27Bv1.7 · 10 s rule75.629.982.7$0.22*Outside: cost 6.7× Jev (v1.5 reference)
  9. –vLLM-SR d3-turbov1.6.43 drawv1.7 · 10 s rule74.829.882.6$0.22*Outside: cost 6.8× Jev (v1.5 reference)
  10. –StartLux-Decision v1.1 27B Instantv1.6.43 drawv1.7 · 10 s rule73.230.482.6$0.21*Outside: cost 6.5× Jev (v1.5 reference)
  11. –vLLM-SR d3v1.6.43 drawv1.7 · 10 s rule72.929.882.1$0.22*Outside: cost 6.8× Jev (v1.5 reference)
  12. –Jeffan-31b-1710v1.7.0 drawv1.7 · 10 s rule71.642.881.7$0.081*Outside: cost 2.5× Jev (v1.5 reference)
  13. –AJev lm6qv1.7.0 drawv1.7 · 10 s rule71.541.081.4$0.093*Outside: cost 2.9× Jev (v1.5 reference)
  14. –Nimble-v4-maxv1.7.0 drawv1.7 · 10 s rule71.030.181.2$0.21*Outside: cost 6.6× Jev (v1.5 reference)
  15. –Turion Jev 31Bv1.7.1 drawv1.7 · 10 s rule70.940.680.9$0.096*Outside: cost 3.0× Jev (v1.5 reference)
  16. –pplx-decider-v1.2v1.6.43 drawv1.7 · 10 s rule71.730.080.8$0.21*Outside: cost 6.6× Jev (v1.5 reference)
  17. –torchcast-decision-27bv1.7 · 10 s rule70.430.380.2$0.21*Outside: cost 6.5× Jev (v1.5 reference)
  18. –Vera Largev1.7.0 drawv1.7 · 10 s rule67.829.979.6$0.22*Outside: cost 6.7× Jev (v1.5 reference)
  19. –Kev 27Bv1.7 · 10 s rule64.218.378.9$0.53*Outside: cost 16.4× Jev (v1.5 reference)
  20. –StartLux-Decision v1.1 27B AutoThinkv1.6.43 drawv1.7 · 10 s rule66.723.578.8$0.35*Outside: cost 11.0× Jev (v1.5 reference)
  21. –SPX-CD ProAPIv1.7 · 10 s rule64.643.178.5$0.079*Outside: cost 2.4× Jev (v1.5 reference)
  22. –Amos 1 largeAPIv1.6.43 drawv1.7 · 10 s rule68.230.078.0$0.21*Outside: cost 6.6× Jev (v1.5 reference)
  23. –Eikos-27Bv1.7 · 10 s rule64.428.777.2$0.24*Outside: cost 7.4× Jev (v1.5 reference)
  24. –Fastino GLiDEAPIv1.7 · 10 s rule66.135.977.0$0.14Outside: cost 4.3× Jev (v1.5 reference)
  25. –wity-1APIv1.7 · 10 s rule67.958.876.9$0.024Outside: latency 3.0× Jev (v1.5 reference)API price · eligibility checked at the developer's list price; at base-model pricing it would exceed the cost cap (2.70× Jev)
  26. –Aleria Judge 1.1.1v1.7.0 drawv1.7 · 10 s rule68.035.876.9$0.14*Outside: cost 4.3× Jev (v1.5 reference)
  27. –KnowLine 35B-A3B-Gen1.5v1.7.0 drawv1.7 · 10 s rule60.324.776.7$0.32*Outside: cost 10.0× Jev (v1.5 reference)
  28. –OpenJevv1.7 · 10 s rule70.228.575.8$0.24*Outside: cost 7.5× Jev (v1.5 reference), latency 3.1× Jev (v1.5 reference)
  29. –Clefv1.7 · 10 s rule59.128.175.3$0.25*Outside: cost 7.7× Jev (v1.5 reference)
  30. –SimpleJev Qwen3.8-27Bv1.7 · 10 s rule57.915.174.9$0.68*Outside: cost 20.9× Jev (v1.5 reference), latency 2.3× Jev (v1.5 reference)
  31. –JEV-27Bv1.7 · 10 s rule56.831.073.9$0.20*Outside: cost 6.2× Jev (v1.5 reference)
  32. –Nagi-ENORMOUS 27Bv1.6.43 drawv1.7 · 10 s rule62.430.073.8$0.22*Outside: cost 6.7× Jev (v1.5 reference)
  33. –RSI-Jev v6.1-VL 27Bv1.6.9 drawv1.7 · 10 s rule61.631.173.5$0.20*Outside: cost 6.1× Jev (v1.5 reference)
  34. –Sage 1.4APIv1.6.43 drawv1.7 · 10 s rule63.759.573.3$0.022*Outside: latency 2.8× Jev (v1.5 reference)
  35. –Autoloops – Gemma 4 31B ITAPIv1.7 · 10 s rule58.140.572.7$0.096Outside: cost 3.0× Jev (v1.5 reference)
  36. –swanOnev1.7 · 10 s rule56.242.272.4$0.085*Outside: cost 2.6× Jev (v1.5 reference)
  37. –FrontiersMind Lumma-fev-27Bv1.7.0 drawv1.7 · 10 s rule55.517.972.3$0.54*Outside: cost 16.9× Jev (v1.5 reference)
  38. –AutoJev-27Bv1.7 · 10 s rule55.629.471.6$0.23*Outside: cost 7.0× Jev (v1.5 reference)
  39. –SimpleJev Qwen3.8-27BAPIv1.7 · 10 s rule52.914.971.6$0.69*Outside: cost 21.3× Jev (v1.5 reference), latency 2.1× Jev (v1.5 reference)
  40. –Decision 2.0 Vega 27Bv1.7 · 10 s rule60.429.671.3$0.22*Outside: cost 6.9× Jev (v1.5 reference)
  41. –JPT-35B-A3Bv1.7 · 10 s rule51.433.670.7$0.16*Outside: cost 5.0× Jev (v1.5 reference)
  42. –Jebadiah 27Bv1.7 · 10 s rule57.529.970.2$0.22*Outside: cost 6.7× Jev (v1.5 reference)
  43. –NInfer Qwen3.8-Flash-Next mixedv1.7 · 10 s rule49.042.569.7$0.082*Outside: cost 2.5× Jev (v1.5 reference)
  44. –Respan Span-02v1.6.43 drawv1.7 · 10 s rule60.945.169.3$0.068*Outside: cost 2.1× Jev (v1.5 reference)
  45. –JevOnev1.7 · 10 s rule46.039.868.9$0.10*Outside: cost 3.1× Jev (v1.5 reference)
  46. –Chon-1.0-E4Bv1.6.43 drawv1.6 · re-measure pending55.070.768.4$0.0094Outside: latency 72.2× Jev (v1.5 reference)
  47. –SimpleJev Qwen3.6-35B-A3BAPIv1.7 · 10 s rule56.335.168.3$0.15*Outside: cost 4.5× Jev (v1.5 reference), latency 2.1× Jev (v1.5 reference)
  48. –NInfer Qwen3.8-27B NVFP4v1.7 · 10 s rule51.829.168.0$0.23*Outside: cost 7.1× Jev (v1.5 reference)
  49. –Open-Jev 27B v1.1v1.7 · 10 s rule52.514.468.0$0.72*Outside: cost 22.2× Jev (v1.5 reference), latency 2.9× Jev (v1.5 reference)
  50. –LitJevv1.7 · 10 s rule43.328.466.6$0.24*Outside: cost 7.6× Jev (v1.5 reference), latency 5.1× Jev (v1.5 reference)
  51. –Tern-35B-A3Bv1.7.0 drawv1.7 · 10 s rule44.142.166.4$0.085*Outside: cost 2.6× Jev (v1.5 reference)
  52. –decider-35b-a3bv1.7 · 10 s rule48.734.466.2$0.15*Outside: cost 4.8× Jev (v1.5 reference)
  53. –Bev / Bonsai 27Bv1.7 · 10 s rule43.828.266.1$0.25*Outside: cost 7.6× Jev (v1.5 reference), latency 3.5× Jev (v1.5 reference)
  54. –Imajev-9Bv1.6.43 drawv1.7 · 10 s rule36.730.862.4$0.20*Outside: cost 6.3× Jev (v1.5 reference)
  55. –Open-Jev 9Bv1.7 · 10 s rule43.933.161.8$0.17*Outside: cost 5.3× Jev (v1.5 reference), latency 2.3× Jev (v1.5 reference)
  56. –JEB-35B-A3Bv1.6.43 drawv1.7 · 10 s rule33.242.961.6$0.080*Outside: cost 2.5× Jev (v1.5 reference)
  57. –Qwen3.5-9B Jev-like data-mix v2v1.7 · 10 s rule41.945.760.9$0.065*Outside: cost 2.001× Jev (v1.5 reference)
  58. –Standard One 8B SHv1.6.9 drawv1.7 · 10 s rule34.543.260.0$0.078*Outside: cost 2.4× Jev (v1.5 reference)
  59. –Gemini 3.1 Flash-LiteAPIv1.7 · 10 s rule58.629.259.8$0.23Outside: cost 7.1× Jev (v1.5 reference)
  60. –reflex 4Bv1.7 · 10 s rule30.863.158.9$0.017*Outside: latency 4.7× Jev (v1.5 reference)
  61. –openjev-sglangAPIv1.7 · 10 s rule38.835.658.7$0.14*Outside: cost 4.3× Jev (v1.5 reference), latency 2.1× Jev (v1.5 reference)
  62. –Standard One 8Bv1.7 · 10 s rule32.843.358.4$0.078*Outside: cost 2.4× Jev (v1.5 reference)
  63. –Clef-omniv1.6.9 drawv1.7 · 10 s rule34.526.858.3$0.28*Outside: cost 8.5× Jev (v1.5 reference)
  64. –Bespoke Nimble 9Bv1.7 · 10 s rule43.136.856.5$0.13*Outside: cost 4.0× Jev (v1.5 reference)
  65. –kev 8Bv1.7 · 10 s rule29.340.450.0$0.097*Outside: cost 3.0× Jev (v1.5 reference)
  66. –Nev-2Bv1.6.43 drawv1.6 · re-measure pending20.271.049.9$0.0092*Outside: latency 20.8× Jev (v1.5 reference)
  67. –JADEv1.7 · 10 s rule47.933.649.2$0.16*Outside: cost 5.1× Jev (v1.5 reference)
  68. –Qwen3.8 27BAPIv1.7 · 10 s rule33.40.048.6$2.18*Outside: cost 67.4× Jev (v1.5 reference), latency 13.5× Jev (v1.5 reference)
  69. –Open-Jev 2Bv1.7 · 10 s rule21.833.147.7$0.17*Outside: cost 5.3× Jev (v1.5 reference)
  70. –Laya typed-decisionsv1.7 · 10 s rule5.582.544.2$0.0038*Outside: latency 3.5× Jev (v1.5 reference)
  71. –Qwen3.5-0.8B Decision Modelv1.6 · re-measure pending7.379.542.6$0.0048*Outside: latency 18.3× Jev (v1.5 reference)
  72. –Gutsy 0.8B v0.3v1.6.9 drawv1.7 · 10 s rule8.678.442.2$0.0053*Outside: latency 3.1× Jev (v1.5 reference)
  73. –Reflex Instinct 0.6Bv1.6.43 drawv1.6 · re-measure pending4.079.838.6$0.0047*Outside: latency 6.3× Jev (v1.5 reference)
  74. –Vega-Uv1.6.43 drawv1.7 · 10 s rule5.379.236.1$0.0049*Outside: latency 2.6× Jev (v1.5 reference)
  75. –open-jev-deberta-v3-largev1.7 · 10 s rule2.877.635.4$0.0056*Outside: latency 5.7× Jev (v1.5 reference)
  76. –modernbert-jev v2v1.6.9 drawv1.6 · re-measure pending2.670.434.6$0.0097*Outside: latency 8.7× Jev (v1.5 reference)
  77. –Layav1.7 · 10 s rule1.884.932.5$0.0032*Outside: latency 3.3× Jev (v1.5 reference)
  78. –SimpleJevv1.7 · 10 s rule12.770.331.1$0.0098*Outside: latency 14.9× Jev (v1.5 reference)
  79. –Raw Qwen3 8B direct logitsv1.7 · 10 s rule26.545.530.1$0.065*Outside: cost 2.02× Jev (v1.5 reference)
  80. –system-onev1.7 · 10 s rule29.345.029.7$0.068*Outside: cost 2.1× Jev (v1.5 reference)
  81. –Mirrorv1.7 · 10 s rule0.089.325.8$0.0023*Outside: latency 2.1× Jev (v1.5 reference)

Jev-class = cost per decision at most 2× Jev's (≤ $0.065 per 1,000 decisions) and median latency at most 2× Jev's (≤ 1.23 s), the adjusted p50 — the same median the speed chart plots, not the four-axis Speed score. 193 of 274 systems qualify; the other 81, including the general-purpose LLMs, are listed below the divider in the ranking. The charts below show speed and cost beside Capability Score; the official JevBench Score weighs all four axes.

Capability against cost and speed

Jev-class systems are shown by default. Bubble size follows the official JevBench Score; official rank stays unchanged. The five most capable Jev-class systems are labelled.

Capability vs cost

Upper right is best: more capable and cheaper. Cost is USD per 1,000 decisions on a log scale. The dashed line is 2× Jev’s cost.

Capability vs cost: 192 systems. Upper right is best: more capable and cheaper.102030405060708090100$0.0010$0.010$0.10$ per 1,000 decisions (log)Capability↑2× Jev (v1.5 reference)← priciercheaper →1. kettei-fat2. Super Decider3. Super Decider4. Fastino GLiDE5. H2O-Lightning-31B v1.0.7
192 systems. Tap a bubble for its values.

Capability vs speed

Upper right is best: more capable and faster. Speed here is the median-latency speed — the same adjusted median (p50) latency the Jev-class limit uses — a log scale, so each 20 points is 10× faster (median under the numbers). Self-hosted systems are plotted at the est. incl. network latency (2x measured + 150 ms); their measured loopback p50 (on-GPU, no network; not directly comparable with API latency) is in the tooltip. The dashed line is 2× Jev’s median latency.

Capability vs speed: 192 systems. Upper right is best: more capable and faster.010203040506070809010070≈3.2 s80≈1.0 s90≈316 ms100≈100 msMedian-latency speedCapability↑2× Jev (v1.5 reference)← slowerfaster →1. kettei-fat2. Super Decider3. Super Decider4. Fastino GLiDE5. H2O-Lightning-31B v1.0.7
192 systems. Tap a bubble for its values.
  • Jev — reference (TypeSafe, closed) (2)
  • Closed API (weights not public) (30)
  • Open weights · LLM decoder (201)
  • Open weights · diffusion LM (6)
  • Open weights · encoder / classifier (21)
  • Open weights · reranker (5)
  • Base model control (no decision fine-tune, raw logits) (5)
  • System (router / cascade / ensemble) (3)
  • faint = outside Jev-class
305 of 305 systems

Model kind

Jev-class

Release

Parameters

No system in this release reports an exact parameter count.

Developer/API price $/1k
Base-model price $/1k
Scored cost $/1k
Alternative price $/1k
p50 latency (s)
p95 latency (s)

JevBench v1.7.2

JevBench Composite Score: 268 ranked systems

Official· four axes 0–100, equal-weight harmonic mean · Method notes ↓

Weights:
Adjust weights ↓

View by:Capability ↑

The official order weighs Intelligence, Calibration, Speed and Cost equally. Each button re-sorts the same systems by one axis.

Greener = stronger within its column.

274 of 274 systems, sorted by official rank, #1 first.

  1. 1Super DeciderAPIfine-tune · 26B-A4B · NVFP4Base model: undisclosed78.5I 79C 91S 85K 64$0.016
  2. 2kettei-fatAPIClosed APIBase model: undisclosed77.7I 80C 93S 93K 57est.$0.026
  3. 3Fastino GLiDE (hosted API, 22 Oct 2026)APIClosed APIBase model: undisclosed*176.4I 76C 93S 87K 59$0.023
  4. 4Torchcast Decision 4B (v1.3)fine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource76.2I 67C 88S 92K 65est.$0.015
  5. 5d1-ultraAPIClosed APIBase model: undisclosed76.0I 70C 91S 89K 62est.$0.018
  6. 6SubQ Snap 1.1APIClosed APIBase model: undisclosed75.7I 76C 88S 85K 61est.$0.020
  7. 7Drex MaxAPIClosed APIBase model: undisclosed*275.4I 79C 90S 90K 55$0.031
  8. 8Bespoke Nimble-v4-maxAPIClosed APIBase model: undisclosed74.8I 68C 91S 88K 61est.$0.020
  9. 9Super Decider (thinking)APIfine-tune · 26B-A4B · NVFP4Base model: undisclosed74.5I 80C 92S 80K 56$0.028
  10. 10SubQ SnapAPIClosed APIBase model: undisclosed74.4I 72C 91S 86K 58est.$0.026
  11. 11H2O-Lightning-4B v1.1fine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource73.8I 60C 90S 92K 64est.$0.016
  12. 12Sage 1.3.0APIClosed APIBase model: undisclosed73.7I 66C 92S 92K 58est.$0.025
  13. 13kettei-chubAPIClosed APIBase model: undisclosed73.4I 62C 84S 95K 63est.$0.018
  14. 14H2O-Lightning-31B v1.0.7fine-tune · 31BBase model: google/gemma-4-31B-itsource73.1I 79C 89S 90K 51est.$0.044
  15. 15Torchcast Decision 4B (v1.2)fine-tune · 4BBase model: undisclosed73.0I 60C 85S 91K 65est.$0.015
  16. 16d1APIClosed APIBase model: undisclosed72.8I 62C 87S 88K 63$0.017
  17. 17Mercury DecideAPISystemBase model: undisclosedsource72.2I 66C 81S 85K 62est.$0.018
  18. 18decisio v0.8.0 on gemma-4-31B-it31B · FP8 on loadBase model: google/gemma-4-31B-itsource71.7I 70C 89S 91K 52est.$0.041
  19. 19basal-omni-1.0-ultramerge · 31BBase model: undisclosed71.5I 74C 90S 87K 50est.$0.045
  20. 20Jev 1.13.0APIJev referenceBase model: undisclosed71.4I 64C 91S 91K 55$0.032
Show all 274 systems (248 more ranked, 6 more not ranked)
  1. Microsoft-Decision-1 (Azure Foundry) — A2 re-run (preliminary)APIPRELIMINARY — API supplement A2 (300 sealed + 300 public), not the S1200 full set; equating pool 4 of 6Closed APIBase model: Qwen3.5-9Bsource · Base model disclosed by Microsoft; the Decision-1 endpoint is evaluated as a hosted API offering.71.4I 59C 85S 89K 62$0.018
  2. 21Quyet-1.0-Largeadapter · 31BBase model: google/gemma-4-31B-itsource71.3I 73C 90S 86K 51est.$0.045
  3. 22Celeris-1 DecisionAPIClosed APIBase model: undisclosed71.0I 59C 88S 87K 60est.$0.021
  4. 23Torchcast Decision 4B (v1.1)fine-tune · 4BBase model: undisclosed70.7I 54C 83S 94K 65est.$0.015
  5. 24Myles-G 31BAPIClosed APIBase model: google/gemma-4-31B-it (self-reported)*370.6I 74C 84S 86K 51est.$0.043
  6. 25Arcbright Elf 1 31Bfine-tune · 31B · bf16Base model: undisclosed70.5I 72C 92S 90K 50est.$0.047
  7. 26decider-12b v2fine-tune · 12BBase model: google/gemma-4-12B-itsource70.3I 62C 81S 90K 58est.$0.026
  8. 27Xevn-4Bfine-tune · 4B · bf16Base model: undisclosed70.2I 52C 85S 92K 66est.$0.014
  9. 28Jev 1.14.0APIJev referenceBase model: undisclosed70.2I 60C 90S 91K 55est.$0.032
  10. 29torchcast-decision-12badapter · 12BBase model: google/gemma-4-12B-itsource69.8I 61C 83S 91K 56est.$0.028
  11. 30decider-12b v1, stock Gemma-4-12B-it12BBase model: google/gemma-4-12B-itsource69.8I 60C 83S 90K 58est.$0.026
  12. 31metask-jev-rain-31B-trainedadapter · 31B · W4A16 INT4Base model: undisclosed69.6I 71C 87S 82K 51est.$0.044
  13. 32vLLM-SR d3-minifine-tune · 4B · bf16Base model: undisclosed69.5I 51C 87S 93K 64est.$0.016
  14. 33Bobcat 1.3fine-tune · 31B · fp8Base model: undisclosed69.4I 68C 88S 89K 50est.$0.047
  15. 34Microsoft-Decision-1 (Azure Foundry)APIClosed APIBase model: Qwen3.5-9Bsource*469.1I 57C 84S 81K 61$0.019
  16. 35wity-1APIClosed APIBase model: undisclosedBase-model reference price (base undisclosed at the author's request): 43.0 (would be #86)69.0I 68C 86S 68K 59$0.024
  17. 36Chon-1.0-12B-Fastadapter · 12B · bf16Base model: undisclosed68.9I 60C 80S 84K 59est.$0.024
  18. 37Winnow-12B Q8merge · 12B · GGUF Q8_0Base model: google/gemma-4-12B-itsource68.8I 60C 83S 86K 57est.$0.028
  19. 38deck-31B31B · FP8Base model: google/gemma-4-31B-itsource68.6I 73C 82S 86K 50est.$0.048
  20. 39Jeff-1.0-Largefine-tune · 31BBase model: undisclosed68.5I 72C 91S 88K 49est.$0.049
  21. 40Cygnet12BBase model: google/gemma-4-12B-itsource68.5I 55C 87S 91K 56est.$0.028
  22. 41Myles-G 12BAPIClosed APIBase model: google/gemma-4-12B-it (self-reported)*568.3I 57C 84S 85K 57est.$0.026
  23. 42Wald 26B-A4Bfine-tune · 26B-A4B · bf16Base model: undisclosed68.2I 66C 88S 92K 50est.$0.048
  24. 43decisio v0.8.0 on gemma-4-12B-it12BBase model: google/gemma-4-12B-itsource68.1I 52C 89S 86K 59est.$0.023
  25. 44attest-31B-v1.0APIbase · 31BBase model: undisclosed67.9I 65C 74S 96K 51est.$0.044
  26. 45Sage 1.4APIClosed APIBase model: undisclosed67.7I 64C 83S 69K 59est.$0.022
  27. 46decisio v0.9.0 on gemma-4-12B-it12BBase model: google/gemma-4-12B-itsource67.7I 52C 88S 85K 59est.$0.023
  28. 47Jev-Omnifine-tuneBase model: google/gemma-4-12B-itsource67.6I 56C 87S 85K 56est.$0.029
  29. 48metask-jev-rain-12B-trainedadapter · 12B · W4A16 INT4Base model: undisclosed67.3I 66C 87S 72K 53est.$0.038
  30. 49Bobcat Flash 1.2fine-tune · 26B / 4B active · FP8 on loadBase model: google/gemma-4-26B-A4B-itsource67.1I 66C 81S 91K 50est.$0.048
  31. 50Xor 26B-A4Bmerge · 26B-A4BBase model: google/gemma-4-26B-A4B-itsource66.8I 57C 85S 91K 51est.$0.044
  32. 51SPX-CD FlashAPISystemBase model: undisclosedsource66.6I 58C 90S 79K 52est.$0.039
  33. 52Messier One v0.4fine-tune · 4BBase model: undisclosed66.1I 52C 73S 89K 61est.$0.020
  34. 53decider chat on Gemma-4-31B-it31BBase model: google/gemma-4-31B-itsource66.0I 59C 82S 86K 51est.$0.044
  35. 54Hopper 12B trainedfine-tune · 12BBase model: google/gemma-4-12B-itsource65.6I 52C 85S 84K 56est.$0.029
  36. 55metask-jev-rain-31B-trained service v3adapter · 31B · W4A16 INT4Base model: undisclosed65.6I 73C 87S 73K 49est.$0.049
  37. 56Surogate Rune 26B-A4B v326B-A4BBase model: google/gemma-4-26B-A4B-itsource*665.0I 56C 91S 87K 50est.$0.048
  38. 57GEV-26B-Decidedistilled · 26B-A4BBase model: google/gemma-4-26B-A4B-itsource64.2I 50C 90S 90K 51est.$0.043
  39. 58SPX-CD-Omnifine-tune · 12BBase model: google/gemma-4-12B-itsource63.1I 49C 87S 89K 58est.$0.025
  40. 59Metask rain 4Bfine-tune · 4BBase model: undisclosed62.5I 51C 81S 78K 52est.$0.040
  41. 60shisa-de-1fine-tune · 26B-A4BBase model: undisclosed62.4I 50C 81S 93K 50est.$0.047
  42. 61OpenAI DecisionsAPIClosed APIBase model: undisclosed62.4I 57C 90S 88K 49$0.052
  43. 62InstinctAPIClosed APIBase model: Qwen3.8-27Bsource*762.2I 48C 93S 87K 64$0.015
  44. 63Blink v0.3 26B-A4Bfine-tune · 26B-A4B · NVFP4Base model: undisclosed61.1I 49C 91S 91K 50est.$0.048
  45. 64HS1 0.1.2APIClosed APIBase model: google/gemma-4-26B-A4B-it (self-reported)*861.1I 53C 76S 94K 49est.$0.050
  46. 65baselabs elephant-1fine-tune · 9B · bf16Base model: undisclosed60.9I 56C 85S 90K 48est.$0.052
  47. 66Diffusion Jev26B / 4B activeBase model: google/diffusiongemma-26B-A4B-itsource59.3I 55C 61S 86K 50est.$0.048
  48. 67René-1 31B FP8fine-tune · 31B · FP8Base model: google/gemma-4-31B-itsource55.8I 62C 90S 87K 46est.$0.063
  49. 68Amazon Strands Decider 26B-A4Badapter · 26B-A4B · bf16Base model: undisclosed54.5I 47C 87S 88K 50est.$0.045
  50. 69Wald 4B v2.1fine-tune · 4BBase model: undisclosed54.3I 52C 87S 93K 46est.$0.062
  51. 70pplx-decider-flash-v1.2fine-tune · 4BBase model: undisclosed54.0I 45C 78S 91K 64est.$0.015
  52. 71Bespoke Nimble 9B v3adapter · 9BBase model: Qwen/Qwen3.5-9Bsource53.5I 57C 77S 89K 46est.$0.063
  53. 72Aplomb 1fine-tune · 5.3BBase model: Qwen/Qwen3.5-4Bsource52.9I 45C 89S 89K 58est.$0.025
  54. 73Respan Span-02LLM decoderBase model: undisclosed51.5I 61C 78S 86K 45est.$0.068
  55. 74Seb-9B-v2fine-tune · 9B · bf16Base model: undisclosed50.0I 46C 84S 92K 49est.$0.051
  56. 75APUS-OpenJev-v1-9Bfine-tune · 9BBase model: Qwen/Qwen3.5-9Bsource49.2I 47C 67S 82K 49est.$0.051
  57. 76Arcbright Elf 1 12Bfine-tune · 12B · bf16Base model: undisclosed48.8I 44C 86S 92K 57est.$0.026
  58. 77Jeffan-31b-1710fine-tune · 31B · bf16Base model: undisclosed48.7I 72C 92S 85K 43est.$0.081
  59. 78Plumb-4Bfine-tune · 4BBase model: alibiserikbay/JevK5source47.9I 43C 88S 94K 63est.$0.017
  60. 79SPX-CD ProAPISystemBase model: undisclosedsource47.6I 65C 92S 77K 43est.$0.079
  61. Chon-1.0-E4B (re-measure pending under the 10 s rule)fine-tune · 4BBase model: undisclosed45.9I 55C 82S 44K 71$0.0094
  62. 80Messier One v0.2fine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource45.3I 42C 74S 93K 65$0.015
  63. 81spark-s1-4b-v6adapter · 4BBase model: Qwen3.5-4Bsource44.8I 43C 65S 86K 60est.$0.021
  64. 82KnowLine 31B-Gen1fine-tune · 31B · bf16Base model: undisclosed44.4I 77C 93S 81K 41est.$0.093
  65. 83swanOneNVFP4Base model: Mia-AiLab/Qwen3.8-Flash-Next-NVFP4source43.8I 56C 89S 83K 42est.$0.085
  66. 84AJev lm6qadapter · 31B · bf16Base model: undisclosed43.7I 71C 91S 82K 41est.$0.093
  67. 85Quyet-1.0-Mediumadapter · 4BBase model: Qwen/Qwen3.5-4Bsource43.6I 42C 77S 89K 63est.$0.017
  68. 86Bespoke Nimble-v4APIClosed APIBase model: undisclosed43.2I 41C 87S 89K 61est.$0.020
  69. 87Turion Jev 31Bfine-tune · 31B · fp8Base model: undisclosed42.9I 71C 91S 86K 41est.$0.096
  70. 88Vansa-3.4APIClosed APIBase model: Qwen3.5-4B (self-reported)*942.6I 41C 87S 95K 62$0.019
  71. 89JPT-4Bmerge · 4BBase model: undisclosed42.2I 41C 87S 88K 64est.$0.016
  72. 90levadapter · 4BBase model: Qwen/Qwen3.5-4Bsource42.1I 41C 82S 88K 62est.$0.019
  73. 91NInfer Qwen3.8-Flash-Next mixedLLM decoderBase model: Qwen3.8-Flash-Nextsource41.9I 49C 90S 89K 43est.$0.082
  74. 92decider-4b v2adapter · 4BBase model: Qwen/Qwen3.5-4B-Basesource41.1I 40C 90S 90K 65est.$0.015
  75. 93baselabs mammoth-1fine-tune · 27B · bf16Base model: undisclosed41.1I 76C 94S 84K 40est.$0.104
  76. 94Jeffan-4B-2010fine-tune · 4B · bf16Base model: undisclosed41.1I 40C 88S 93K 64est.$0.016
  77. 95jev-local9BBase model: Qwen/Qwen3.5-9Bsource41.0I 42C 73S 73K 59est.$0.024
  78. 96GPT-6 Luna (low reasoning effort)APIClosed APIBase model: undisclosed40.4I 96C 98S 72K 39$0.108
  79. 97Autoloops – Gemma 4 31B ITAPIClosed APIBase model: undisclosed40.2I 58C 87S 84K 40$0.096
  80. 98Wald 4Bfine-tune · 4BBase model: undisclosed39.7I 41C 78S 82K 54est.$0.033
  81. 99GPT-6 Luna (default medium reasoning effort)APIClosed APIBase model: undisclosed39.1I 97C 100S 71K 38$0.113
  82. 100ClassOne Qwen 3.5 9B9BBase model: Qwen/Qwen3.5-9Bsource38.5I 42C 59S 90K 49est.$0.049
  83. 101JevK5 v0.3adapterBase model: Qwen/Qwen3.5-4Bsource37.4I 39C 90S 94K 63est.$0.017
  84. 102metask-jev-4b4BBase model: Qwen3.5-4Bsource37.3I 39C 84S 90K 58est.$0.026
  85. 103deck-4B v1.0adapter · FP8Base model: alibiserikbay/JevK5 (built on Qwen/Qwen3.5-4B)source 1source 237.3I 39C 89S 90K 63est.$0.017
  86. 104Kahn1 4Bfine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource37.0I 40C 84S 89K 50est.$0.046
  87. 105Clef-Flashfine-tuneBase model: Qwen/Qwen3.5-9BsourceWorkers AI price scenario (latency unmeasured): 39.1 (would be #99)36.6I 42C 85S 88K 47est.$0.059
  88. 106Decision-4BadapterBase model: Qwen/Qwen3.5-4Bsource35.4I 38C 80S 91K 66est.$0.014
  89. 107decisor-4bfine-tune · 4B · FP8Base model: Qwen/Qwen3.5-4Bsource35.3I 40C 52S 92K 64est.$0.016
  90. 108janus 4Bfine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource35.2I 38C 88S 93K 64est.$0.015
  91. 109Qwen3.5-9B Jev-like data-mix v2adapter · 9BBase model: Qwen/Qwen3.5-9Bsource33.6I 42C 80S 85K 46est.$0.065
  92. 110Seb-9Bfine-tune · 9B · bf16Base model: Qwen/Qwen3.5-9Bsource32.6I 38C 88S 91K 49est.$0.051
  93. 111JevK5 v0.2.0adapter · 4BBase model: Qwen3.5-4Bsource32.1I 36C 89S 91K 63est.$0.017
  94. 112Tern-35B-A3Bfine-tune · 35B-A3B · bf16Base model: undisclosed32.1I 44C 89S 91K 42est.$0.085
  95. 113Amos 1 mediumAPI9BBase model: undisclosed31.6I 38C 88S 86K 49est.$0.051
  96. 114Aleria Judge 1.1.131B · bf16Base model: undisclosed31.2I 68C 86S 86K 36est.$0.138
  97. 115JevOneadapterBase model: Qwen/Qwen3.6-35B-A3Bsource31.1I 46C 92S 90K 40est.$0.101
  98. 116Fastino GLiDEAPIClosed APIBase model: undisclosed30.6I 66C 88S 77K 36$0.137
  99. 117Jebadiah 9B v2.1merge · 9BBase model: undisclosed30.3I 37C 90S 91K 49est.$0.051
  100. 118Verdin-4Bmerge · 4BBase model: undisclosed29.9I 35C 84S 90K 62est.$0.018
  101. 119TypeCastLM 1.4.03.8BBase model: Qwen/Qwen3.5-4Bsource29.8I 35C 77S 93K 65est.$0.015
  102. 120blink-4bmerge · 4BBase model: undisclosed29.5I 35C 88S 91K 64est.$0.016
  103. 121jiwo 4bfine-tune · 4.2BBase model: undisclosed28.9I 35C 86S 92K 64est.$0.016
  104. 122Imajev-4Badapter · 4BBase model: Qwen/Qwen3.5-4Bsource28.6I 35C 89S 90K 63est.$0.017
  105. 123SimpleJev Qwen3.6-35B-A3BAPI35B-A3BBase model: undisclosedsource27.5I 56C 80S 76K 35est.$0.145
  106. 124Uniphore-4B-Deciderfine-tune · 4BBase model: undisclosed27.2I 34C 80S 87K 65est.$0.015
  107. 125Decision 2.0 Lux 9BLLM decoderBase model: undisclosed27.1I 36C 87S 90K 48est.$0.052
  108. 126Hopperadapter · 4BBase model: Qwen/Qwen3.5-4Bsource26.5I 34C 89S 91K 62est.$0.018
  109. 127Malkuth-4Badapter · 4BBase model: Qwen/Qwen3.5-4B-Basesource26.2I 34C 88S 89K 56est.$0.030
  110. 128jqv32BBase model: Qwen3-32Bsource25.4I 34C 90S 82K 51est.$0.042
  111. 129JPT-35B-A3Bmerge · 35B-A3BBase model: Qwen/Qwen3.5-35B-A3Bsource25.4I 51C 90S 90K 34est.$0.163
  112. 130decider-35b-a3bfine-tune · 35B-A3BBase model: Qwen/Qwen3.5-35B-A3B-Basesource24.8I 49C 84S 91K 34est.$0.154
  113. 131CoCo-Decision-4Bfine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource24.6I 33C 81S 91K 65est.$0.014
  114. 132Manchego v2.1adapter · 4BBase model: Qwen/Qwen3.5-4Bsource24.5I 33C 84S 91K 64est.$0.015
  115. 133Decision 4B v1.2adapter · 4BBase model: Qwen/Qwen3.5-4Bsource24.5I 32C 88S 94K 63est.$0.017
  116. 134Standard One 3B SH v4fine-tune · 3B · bf16Base model: undisclosed24.4I 34C 78S 93K 49est.$0.052
  117. 135Fastino GLiDE 1.2fine-tune · 27B · bf16Base model: undisclosed22.0I 76C 93S 85K 31est.$0.205
  118. 136Raw Qwen3 4B Instruct 2507 direct logitsoriginal · 4BBase model: Qwen/Qwen3-4B-Instruct-2507 (unmodified)source21.8I 34C 35S 91K 63est.$0.017
  119. 137StartLux-Decision v1.1 27B Instantfine-tune · 27BBase model: undisclosed21.6I 73C 92S 90K 30est.$0.209
  120. 138Standard One 3B SHfine-tune · 3BBase model: undisclosed21.6I 33C 78S 86K 49est.$0.052
  121. 139RSI-Jev v6.1-VL 27Bmerge · 27BBase model: undisclosed21.6I 62C 85S 85K 31est.$0.197
  122. 140GPT-5.6 LunaAPIClosed APIBase model: undisclosed21.4I 93C 94S 73K 30$0.213
  123. 141JEV-27Bdistilled · 27BBase model: Qwen/Qwen3.8-27Bsource21.3I 57C 91S 86K 31est.$0.199
  124. 142torchcast-decision-27bmerge · 27BBase model: startlux-models/StartLux-Decision-27Bsource 1source 2*1021.1I 70C 90S 88K 30est.$0.210
  125. 143Bespoke Nimble 9Badapter · 9BBase model: Qwen3.5-9Bsource21.1I 43C 70S 86K 37est.$0.128
  126. 144Nimble-v4-maxfine-tune · 27BBase model: undisclosed20.7I 71C 91S 85K 30est.$0.214
  127. 145pplx-decider-v1.2fine-tune · 27BBase model: undisclosed20.6I 72C 90S 86K 30est.$0.215
  128. 146Perplexity Decider v1.1 27Bfine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource20.6I 76C 90S 86K 30est.$0.217
  129. 147vLLM-SR d3-turbofine-tune · 27BBase model: undisclosed20.5I 75C 90S 86K 30est.$0.218
  130. 148vLLM-SR d3fine-tune · 26BBase model: undisclosed20.5I 73C 91S 87K 30est.$0.218
  131. 149reflex 4Badapter · 4BBase model: Qwen/Qwen3.5-4Bsource20.4I 31C 87S 69K 63est.$0.017
  132. 150Vera Largefine-tune · 27B · bf16Base model: undisclosed20.3I 68C 91S 88K 30est.$0.217
  133. 151Amos 1 largeAPI27BBase model: undisclosed20.3I 68C 88S 83K 30est.$0.215
  134. 152JADEadapter · 27BBase model: Qwen/Qwen3.8-27Bsource20.2I 48C 51S 85K 34est.$0.164
  135. 153Nagi-ENORMOUS 27Badapter · 27BBase model: undisclosed19.8I 62C 85S 86K 30est.$0.216
  136. 154typecastlm4BBase model: Qwen3.5-4Bsource19.7I 30C 83S 93K 64est.$0.016
  137. 155Jebadiah 27Bmerge · 27BBase model: Qwen/Qwen3.8-27Bsource19.2I 57C 83S 85K 30est.$0.217
  138. 156iota-decide-4bfine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4B-Basesource18.9I 29C 85S 87K 63est.$0.017
  139. 157Decision 2.0 Vega 27Badapter · 27BBase model: Qwen/Qwen3.8-27Bsource 1source 218.9I 60C 82S 83K 30est.$0.222
  140. 158Jebadiah 9B v2merge · 9BBase model: undisclosed18.8I 31C 83S 91K 49est.$0.051
  141. 159Standard One 8B SHfine-tune · 8BBase model: undisclosed18.7I 34C 85S 84K 43est.$0.078
  142. 160Decision 4B v1.1adapter · 4BBase model: Qwen/Qwen3.5-4Bsource18.7I 29C 85S 94K 63est.$0.017
  143. 161AutoJev-27Bfine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource18.5I 56C 88S 88K 29est.$0.226
  144. 162pjev-2bfine-tune · 2.5BBase model: undisclosed18.1I 29C 84S 93K 64est.$0.016
  145. 163Eikos-27Badapter · 27BBase model: Qwen/Qwen3.8-27Bsource18.1I 64C 90S 88K 29est.$0.238
  146. 164Instinct Dual 4BAPIClosed APIBase model: Qwen3.5-4Bsource*1118.0I 28C 86S 89K 75$0.0068
  147. 165NInfer Qwen3.8-27B NVFP427BBase model: Qwen3.8-27Bsource17.7I 52C 84S 90K 29est.$0.231
  148. 166Compass 0.2.0adapter · 4BBase model: undisclosed17.6I 29C 83S 83K 58est.$0.026
  149. 167JEB-35B-A3Bfine-tune · 35B-A3BBase model: undisclosed17.3I 33C 90S 93K 43est.$0.080
  150. 168OpenJev26B-A4BBase model: google/diffusiongemma-26B-A4B-itsource 1source 217.3I 70C 81S 73K 29est.$0.241
  151. 169Open-Jev 9Badapter · 9BBase model: Qwen/Qwen3.5-9Bsource17.0I 44C 80S 74K 33est.$0.170
  152. 170Gemini 3.1 Flash-LiteAPIClosed APIBase model: undisclosed16.9I 59C 61S 79K 29$0.230
  153. 171Standard One 8Badapter · 8BBase model: mistralai/Ministral-3-8B-Instruct-2512-BF16source16.9I 33C 84S 92K 43est.$0.078
  154. 172Cleffine-tuneBase model: Qwen/Qwen3.8-27BsourceWorkers AI price scenario (latency unmeasured): 29.4 (would be #121)16.7I 59C 92S 83K 28est.$0.249
  155. 173ArseneLupin v1.1adapter · 4BBase model: undisclosed16.5I 28C 82S 90K 61est.$0.020
  156. 174system-one-openAPIadapter · 2BBase model: Gemma 4 E2Bsource16.4I 28C 76S 79K 68est.$0.011
  157. 175JEV Qwen3.5-9B Base NVFP49B · NVFP4Base model: ig1/Qwen3.5-9B-NVFP4source16.0I 29C 85S 93K 48est.$0.056
  158. 176Arbiter-4Bfine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4Bsource15.9I 27C 77S 92K 66est.$0.013
  159. 177Decider-4Bfine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4Bsource15.7I 27C 87S 92K 66est.$0.013
  160. 178ClassOne Gemma 4 E4Bfine-tune · E4BBase model: undisclosed15.6I 28C 49S 92K 69est.$0.011
  161. 179SemIf4BBase model: Qwen/Qwen3.5-4Bsource15.5I 27C 84S 90K 63est.$0.017
  162. 180openjev-sglangAPI35B-A3BBase model: Qwen/Qwen3.6-35B-A3Bsource15.4I 39C 79S 78K 36est.$0.140
  163. 181DecisionNode Edge 0.9.2fine-tune · 2B · bf16Base model: undisclosed15.3I 27C 77S 93K 71est.$0.0091
  164. 182RYOTIDE-Qwen9pipeline · 9B · BF16Base model: Qwen/Qwen3.5-9Bsource14.7I 28C 77S 86K 49est.$0.049
  165. 183ClassOne Qwen 3.5 4Bfine-tune · 4BBase model: undisclosed14.3I 27C 49S 91K 65est.$0.015
  166. 184Jez-large-1.0.0fine-tune · E4B · bf16Base model: undisclosed13.8I 26C 83S 91K 71est.$0.0096
  167. 185RYOTIDE-Qwenpipeline · 4BBase model: undisclosed12.9I 25C 87S 89K 62est.$0.018
  168. 186Chon-1.0-12Bfine-tune · 12BBase model: undisclosed12.8I 27C 59S 65K 49est.$0.050
  169. 187NomaLLM decoderBase model: undisclosed12.7I 25C 84S 93K 62est.$0.019
  170. 188KnowLine 35B-A3B-Gen1.5fine-tune · 35B-A3B · fp8Base model: undisclosed12.4I 60C 93S 92K 25est.$0.323
  171. 189Bev / Bonsai 27B27B · GGUF PQ2_0Base model: Qwen/Qwen3.8-27Bsource11.7I 44C 89S 73K 28est.$0.247
  172. 190LitJev27BBase model: Qwen/Qwen3.8-27Bsource11.5I 43C 90S 68K 28est.$0.244
  173. 191local-jev Qwen3.5-4Bfine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource11.4I 24C 86S 86K 59est.$0.023
  174. 192system-one8BBase model: Qwen3-8Bsource11.0I 29C 30S 91K 45est.$0.068
  175. 193StartLux-Decision v1.1 27B AutoThinkfine-tune · 27BBase model: undisclosed10.8I 67C 91S 75K 24est.$0.354
  176. 194kev 8Badapter · 8BBase model: Qwen/Qwen3-8B-Basesource10.6I 29C 71S 88K 40est.$0.097
  177. 195Decision 2Badapter · 2BBase model: openbmb/MiniCPM5-2Bsource10.3I 23C 87S 90K 66est.$0.013
  178. 196OpenSourceJevadapter · 4B · GGUF Q4_K_MBase model: Qwen3.5-4Bsource10.2I 23C 74S 76K 69est.$0.011
  179. 197Imajev-9Badapter · 9BBase model: undisclosed9.8I 37C 88S 79K 31est.$0.203
  180. 198open-alternative-jev4BBase model: Qwen3.5-4Bsource9.4I 22C 76S 91K 63est.$0.017
  181. 199Raw Qwen3 8B direct logitsoriginal · 8BBase model: Qwen/Qwen3-8B-Basesource9.3I 27C 34S 88K 46est.$0.065
  182. 200Liquid AI d1-3Bfine-tune · 3BBase model: undisclosed8.9I 22C 77S 94K 51est.$0.044
  183. 201Krino-2Badapter · 2BBase model: undisclosed8.7I 22C 63S 91K 61est.$0.020
  184. 202Imajev-2Badapter · 2BBase model: undisclosed7.9I 21C 87S 81K 52est.$0.041
  185. 203decider-2badapter · 2BBase model: Qwen/Qwen3.5-2B-Basesource7.7I 21C 69S 94K 65est.$0.015
  186. 204Nemotron Diffusion 8B8BBase model: nvidia/Nemotron-Labs-Diffusion-8Bsource7.3I 20C 75S 94K 55est.$0.030
  187. Nev-2B (re-measure pending under the 10 s rule)distilled · 2B · GGUF Q8_0Base model: undisclosed6.9I 20C 80S 55K 71est.$0.0092
  188. 205DeepSeek V4.1 FlashAPILLM decoderBase model: undisclosed6.9I 84C 95S 66K 20$0.480
  189. 206kev 4Badapter · 4BBase model: Qwen/Qwen3-4B-Basesource*126.6I 20C 68S 90K 66est.$0.014
  190. 207Clef-omnifine-tune · 30B-A3BBase model: undisclosed6.1I 34C 82S 87K 27est.$0.276
  191. 208Kev 27Bfine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource5.8I 64C 94S 83K 18est.$0.529
  192. 209FrontiersMind Lumma-fev-27Bfine-tune · 27B · bf16Base model: undisclosed5.3I 55C 89S 85K 18est.$0.544
  193. 210DecisionNode Edge Mini 0.9.2fine-tune · 0.8B · bf16Base model: undisclosed5.3I 18C 78S 93K 80est.$0.0046
  194. 211jiwo 0.8bfine-tune · 0.75BBase model: undisclosed4.1I 16C 79S 94K 78est.$0.0052
  195. 212Malkuth-2Badapter · 2BBase model: empero-ai/Qwen3.8-2B-Distillsource4.0I 16C 79S 92K 66est.$0.014
  196. 213Qwen3-Reranker-4BRerankerBase model: Qwen/Qwen3-4B-Basesource3.7I 16C 73S 81K 48est.$0.052
  197. 214SimpleJev Qwen3.8-27B (self-hosted, v1.6 pool)27BBase model: Qwen/Qwen3.8-27Bsource3.4I 58C 92S 76K 15est.$0.676
  198. 215ZeroEntropy zerank-2RerankerBase model: Qwen/Qwen3-4Bsource3.3I 16C 83S 82K 48est.$0.052
  199. 216EXAONE-4.0-1.2B-JEV v0.31.2BBase model: LGAI-EXAONE/EXAONE-4.0-1.2Bsource3.3I 15C 81S 94K 58est.$0.025
  200. 217Open-Jev 2Badapter · 2BBase model: Qwen/Qwen3.5-2Bsource3.2I 22C 74S 77K 33est.$0.170
  201. 218SimpleJev Qwen3.8-27BAPI27BBase model: Qwen/Qwen3.8-27Bsource3.2I 53C 90S 76K 15est.$0.687
  202. 219Vega 4Badapter · 4BBase model: undisclosed3.1I 16C 57S 84K 47est.$0.058
  203. 220Open-Jev 27B v1.1adapter · 27BBase model: Qwen/Qwen3.8-27Bsource2.9I 52C 83S 72K 14est.$0.716
  204. 221Raw Phi-4 mini direct logitsoriginalBase model: microsoft/Phi-4-mini-instruct (unmodified)source2.5I 14C 57S 92K 53est.$0.036
  205. 222decision-machine-1APIClosed APIBase model: undisclosed2.4I 13C 79S 93K 58$0.024
  206. 223SimpleJev0.8BBase model: Qwen/Qwen3.5-0.8Bsource2.0I 13C 49S 59K 70est.$0.0098
  207. 224Jeetu Decider 2Bnewfine-tune · 2B · bf16Base model: undisclosed1.2I 10C 73S 91K 70est.$0.0098
  208. 225JevActAPIClosed APIBase model: undisclosedsource1.1I 10C 47S 76K 69est.$0.011
  209. 226smalljev semantic-v9adapter · 2BBase model: openbmb/MiniCPM5-2B-Basesource0.8I 9C 77S 90K 61est.$0.020
  210. 227Gutsy 0.8B v0.3fine-tune · 0.8B · GGUF Q8_0Base model: undisclosed0.8I 9C 76S 72K 78est.$0.0053
  211. 228Vega-U (Qwen3.5-4B reader)adapter · 4BBase model: undisclosed0.7I 8C 77S 89K 65est.$0.015
  212. 229PurpleMIST-Mini-1.0fine-tune · 2BBase model: undisclosed0.7I 8C 78S 89K 71est.$0.0094
  213. 230kev 0.6Badapter · 0.6BBase model: Qwen/Qwen3-0.6B-Basesource0.5I 8C 66S 91K 80est.$0.0046
  214. 231jul fast2BBase model: openbmb/MiniCPM5-2Bsource0.5I 7C 78S 91K 69est.$0.011
  215. 232OpenDecisionEncoder / classifierBase model: MoritzLaurer/ModernBERT-large-zeroshot-v2.0source0.5I 7C 69S 86K 79est.$0.0050
  216. 233ClassOne Gemma 4 E2BE2BBase model: google/gemma-4-E2B-itsource0.5I 8C 23S 93K 70est.$0.0098
  217. Qwen3.5-0.8B Decision Model (re-measure pending under the 10 s rule)fine-tune · 0.8BBase model: Qwen/Qwen3.5-0.8B-Basesource0.5I 7C 78S 56K 80est.$0.0048
  218. 234Fastino GLiNER-2.5-DecideAPIfine-tune · 340MBase model: fastino/gliner2-large-v1source 1source 2*130.3I 7C 62S 85K 46$0.064
  219. 235Deem 0.8B v1LLM decoderBase model: Qwen/Qwen3.5-0.8Bsource0.3I 7C 30S 85K 80est.$0.0045
  220. 236Decision Fastadapter · 0.6BBase model: Qwen/Qwen3-0.6B-Basesource0.3I 6C 75S 91K 80est.$0.0046
  221. 237Quyet-1.0-Small-ENfine-tune · 153MBase model: answerdotai/ModernBERT-basesource0.3I 6C 73S 95K 89est.$0.0023
  222. 238ClassOne Qwen 3.5 2Bfine-tune · 2BBase model: undisclosed0.2I 6C 33S 93K 70est.$0.0098
  223. 239Laya typed-decisionsfine-tune · 421MBase model: ModernBERT-largesource0.2I 5C 83S 69K 83est.$0.0038
  224. 240Vega-U (Qwen3.5-0.8B reader)adapter · 0.8BBase model: undisclosed0.2I 5C 67S 73K 79est.$0.0049
  225. 241WaterSheepEncoder / classifierBase model: answerdotai/ModernBERT-basesource0.2I 5C 66S 77K 69est.$0.011
  226. 242openJev Verdict 1.4Encoder / classifierBase model: knowledgator/gliclass-modern-base-v2.0source0.2I 5C 76S 81K 87est.$0.0028
  227. 243Raw Qwen3 0.6B direct logitsoriginal · 0.6BBase model: Qwen/Qwen3-0.6B-Basesource0.2I 6C 11S 93K 78est.$0.0056
  228. 244lev-350madapterBase model: LiquidAI/LFM2.5-350Msource0.2I 5C 82S 94K 80est.$0.0046
  229. 245Raw Qwen3 1.7B direct logitsoriginal · 1.7BBase model: Qwen/Qwen3-1.7B-Basesource0.1I 5C 12S 93K 69est.$0.011
  230. 246kev 0.5Badapter · 0.5BBase model: Qwen/Qwen2.5-0.5Bsource0.1I 5C 61S 92K 80est.$0.0046
  231. 247Jez-small-1.0.0fine-tune · 0.8B · fp32Base model: undisclosed0.1I 5C 82S 93K 80est.$0.0048
  232. 248openJev Verdictfine-tuneBase model: knowledgator/gliclass-modern-base-v2.0source 1source 20.1I 4C 49S 78K 87est.$0.0028
  233. 249Quyet-1.0-Smallfine-tune · 328MBase model: aisingapore/SEA-LION-ModernBERT-300Msource0.1I 4C 61S 95K 80est.$0.0048
  234. Reflex Instinct 0.6B (re-measure pending under the 10 s rule)adapter · 0.6BBase model: undisclosed0.1I 4C 73S 59K 80est.$0.0047
  235. 250watt-flash-0.1fine-tune · 140MBase model: jhu-clsp/mmBERT-smallsource0.1I 4C 79S 74K 89$0.0023
  236. 251Quyet-1.0-Tinydistilled · 183MBase model: jhu-clsp/mmBERT-smallsource0.1I 4C 58S 95K 88est.$0.0024
  237. 252Phocinae-Largha-150M-v1fine-tune · 144MBase model: undisclosed0.1I 4C 57S 77K 95est.$0.0014
  238. 253Liquid AI d1-omni-600Mfine-tune · 600MBase model: undisclosed0.0I 3C 62S 94K 73est.$0.0079
  239. 254Vega 0.8Badapter · 0.8BBase model: undisclosed0.0I 3C 63S 89K 61est.$0.019
  240. 255open-jev-deberta-v3-largeadapterBase model: microsoft/deberta-v3-largesource0.0I 3C 68S 67K 78est.$0.0056
  241. modernbert-jev v2 (re-measure pending under the 10 s rule)fine-tune · 150M · ONNX INT8Base model: undisclosed0.0I 3C 67S 58K 70est.$0.0097
  242. 256Julia-1fine-tune · 144MBase model: undisclosed0.0I 2C 17S 78K 90est.$0.0022
  243. 257verdict-smallfine-tune · 118MBase model: intfloat/multilingual-e5-smallsource0.0I 2C 73S 89K 100est.$0.0009
  244. 258Tacet Sonatafine-tune · 144MBase model: jhu-clsp/mmBERT-smallsource0.0I 2C 46S 79K 89est.$0.0023
  245. 259Layafine-tune · 421MBase model: ModernBERT-largesource0.0I 2C 63S 73K 85est.$0.0032
  246. 260Mixedbread mxbai-rerank-base-v2RerankerBase model: Qwen2.5 (size not stated)source 1source 20.0I 1C 87S 92K 60est.$0.021
  247. 261Laya multilingualfine-tune · 322MBase model: mmBERT-basesource0.0I 0C 34S 77K 82est.$0.0039
  248. 262Certo v1Encoder / classifierBase model: ModernBERT-largesource0.0I 0C 86S 92K 97est.$0.0013
  249. 263CLM-8Bfine-tune · 8BBase model: Qwen/Qwen3-8Bsource0.0I 0C 40S 92K 51est.$0.045
  250. 264BAAI bge-reranker-v2-m3fine-tuneBase model: BAAI/bge-m3source0.0I 0C 88S 94K 59est.$0.023
  251. 265Alibaba GTE Reranker ModernBERT-basefine-tuneBase model: answerdotai/ModernBERT-basesource0.0I 0C 83S 94K 69est.$0.011
  252. 266MirrorEncoder / classifierBase model: undisclosed0.0I 0C 52S 74K 89est.$0.0023
  253. 267Open Jev JSON Canvas26B-A4BBase model: google/diffusiongemma-26B-A4B-itsource0.0I 61C 0S 85K 49est.$0.049
  254. 268Qwen3.8 27BAPI27BBase model: undisclosed0.0I 33C 64S 55K 0est.$2.178

Base-model notes

  1. *1 Fastino GLiDE (base model: undisclosed): No public base model declared; proprietary vendor-hosted API.
  2. *2 Drex Max (base model: undisclosed): No public base model declared; proprietary vendor-hosted API.
  3. *3 Myles-G 31B (base model: google/gemma-4-31B-it (self-reported)): Customer statement (order notes, 21 Oct 2026): a merged LoRA adapter on google/gemma-4-31B-it. Not publicly documented by the vendor and not independently verified.
  4. *4 Microsoft-Decision-1 (base model: Qwen3.5-9B): Base model disclosed by Microsoft; the Decision-1 endpoint is evaluated as a hosted API offering.
  5. *5 Myles-G 12B (base model: google/gemma-4-12B-it (self-reported)): Customer states merged LoRA on Gemma 4 12B; not independently verified.
  6. *6 Surogate Rune 26B-A4B v3 (base model: google/gemma-4-26B-A4B-it): This model also powers our System1 Models s1-pro service.
  7. *7 Instinct (base model: Qwen3.8-27B): Operator-reported; weights not publicly verifiable.
  8. *8 HS1 0.1.2 (base model: google/gemma-4-26B-A4B-it (self-reported)): Customer statement (order notes, 21 Oct 2026): a LoRA fine-tune of Gemma-4-26B-A4B-it served in FP8. Not publicly documented and not independently verified.
  9. *9 Vansa-3.4 (base model: Qwen3.5-4B (self-reported)): Developer-reported to Benchmark Heaven (private correspondence, 1 Oct 2026): built on Qwen3.5-4B via an open decision fine-tune, with Vansa's own adapters; the intermediate fine-tune is not named. Not publicly documented or independently verified.
  10. *10 torchcast-decision-27b (base model: startlux-models/StartLux-Decision-27B): Direct base is StartLux-Decision-27B (startlux-models, CC-BY-NC-4.0), itself a modified Alibaba Qwen model; its card does not name the Qwen version. Qwen3.8-27B is inferred from the identical text_config.
  11. *11 Instinct Dual 4B (base model: Qwen3.5-4B): Operator-reported; weights not publicly verifiable.
  12. *12 kev 4B (base model: Qwen/Qwen3-4B-Base): Evaluated Qwen3 variant; later releases use a different base.
  13. *13 Fastino GLiNER-2.5-Decide (base model: fastino/gliner2-large-v1): Measured via Fastino's hosted API. A 340M English classifier (8k context), not built for multi-step reasoning: near chance on Noul and Score items; 23 very long items exceed its context. Our request follows Fastino's documented format. The open checkpoint scores 11.2 on the community Decision Index (rank 53 of 70; Jev 57.9). Fastino's larger GLiDE is a separate model, ranked separately on the API board.
Weights:
Adjust weights ↓

Weights are relative: each axis counts in proportion to its slider. The score stays a weighted harmonic mean with the low-axis gates; the gates still apply when an axis sits at 0. Only equal weights give the official JevBench Score and rank.

Score = 4 / (1/I + 1/C + 1/S + 1/K) (each 0–100; × (axis / 50)² for Intelligence, Speed or Cost below 50)

  • Jev — reference (TypeSafe, closed) (2)
  • Closed API (weights not public) (31)
  • Open weights · LLM decoder (201)
  • Open weights · diffusion LM (6)
  • Open weights · encoder / classifier (21)
  • Open weights · reranker (5)
  • Base model control (no decision fine-tune, raw logits) (5)
  • System (router / cascade / ensemble) (3)
  • Striped bar = same system under the labelled alternative price assumption
  • Shown, not ranked
I, C, S, K = Intelligence, Calibration, Speed, Cost; the est. pill = estimated cost; ann. = announced price; API = the operator's endpoint saw held-out benchmark inputs, without answers; $/1k decisions = US dollars per 1,000 decisions (not heat-shaded). Names link to each project.

Compare two systems

Pick any two. Radars for the score axes, capability by subject topic and use cases, chance-corrected competence per request type on the open and sealed sets, and competence per tier on each set. Further out is better on every spoke; the link keeps the pair.

  • A: Jev 1.13.0REFERENCE · TypeSafe — Jev — reference (TypeSafe, closed) · Score 71.4 (#20)
  • B: Super Decider — Open weights · diffusion LM · Score 78.5 (#1)

The four score axes

Radar: the four score axes, two systemsThe four score axes, Jev 1.13.0 vs Super Decider. Intelligence: 63.6 vs 78.9; Calibration: 90.6 vs 90.8; Speed: 91.2 vs 85.3; Cost: 54.7 vs 64.3.50100Intelligence: Jev 1.13.0: 63.6; Super Decider: 78.9Intelligence63.6 · 78.9Calibration: Jev 1.13.0: 90.6; Super Decider: 90.8Calibration90.6 · 90.8Speed: Jev 1.13.0: 91.2; Super Decider: 85.3Speed91.2 · 85.3Cost: Jev 1.13.0: 54.7; Super Decider: 64.3Cost54.7 · 64.3
0–100, the values in the table. An axis a system has no published value for is left as a gap (it counts as 0 in the composite).

Capability by subject topic

Super Decider: No per-category values are published for this system.

Radar: capability by subject topic, two systemsCapability by subject topic, Jev 1.13.0 vs Super Decider. Math & numbers: 40.5 vs —; Coding & software: 80.3 vs —; Rules, policy & law: 73.7 vs —; Finance & commerce: 83.4 vs —; Support & operations: 91.5 vs —; Everyday language: 96.1 vs —; Safety & security: 67.7 vs —. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Math & numbers: Jev 1.13.0: 40.5; Super Decider: No measured result — Math & numbers: a calculation decides the answer: arithmetic, word problems, probability, dates, units. 235 items (174 open, 61 sealed).Math &numbers40.5 · n/aCoding & software: Jev 1.13.0: 80.3; Super Decider: No measured result — Coding & software: code, SQL, repositories, developer tools and IT systems. 92 items (67 open, 25 sealed).Coding &software80.3 · n/aRules, policy & law: Jev 1.13.0: 73.7; Super Decider: No measured result — Rules, policy & law: applying written rules: company policies, contracts, regulations, eligibility. 548 items (260 open, 288 sealed).Rules, policy& law73.7 · n/aFinance & commerce: Jev 1.13.0: 83.4; Super Decider: No measured result — Finance & commerce: money: payments, refunds, invoices, orders, expenses, insurance payouts. 224 items (134 open, 90 sealed).Finance &commerce83.4 · n/aSupport & operations: Jev 1.13.0: 91.5; Super Decider: No measured result — Support & operations: support tickets, incidents, logistics, scheduling desks and routing work to a team. 282 items (131 open, 151 sealed).Support &operations91.5 · n/aEveryday language: Jev 1.13.0: 96.1; Super Decider: No measured result — Everyday language: short everyday messages: intents, assistant requests, reading a detail out of a text. 115 items (98 open, 17 sealed).Everydaylanguage96.1 · n/aSafety & security: Jev 1.13.0: 67.7; Super Decider: No measured result — Safety & security: untrusted or injected instructions, fraud, moderation, access and security triage. 128 items (40 open, 88 sealed).Safety &security67.7 · n/a

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

What the decision is about — math and dates, coding, rules, money, support work, everyday messages, safety. Chance-corrected competence per category (0 = chance, below chance negative, 100 = perfect), Jev 1.13.0: open+sealed; Super Decider: open+sealed items pooled; only categories with at least 30 items are spokes, smaller ones are listed below. Hover a category for its definition and item count.
What each category means · items per category
  • Math & numbers — a calculation decides the answer: arithmetic, word problems, probability, dates, units. 235 items (174 open / 61 sealed)
  • Coding & software — code, SQL, repositories, developer tools and IT systems. 92 items (67 open / 25 sealed)
  • Rules, policy & law — applying written rules: company policies, contracts, regulations, eligibility. 548 items (260 open / 288 sealed)
  • Finance & commerce — money: payments, refunds, invoices, orders, expenses, insurance payouts. 224 items (134 open / 90 sealed)
  • Support & operations — support tickets, incidents, logistics, scheduling desks and routing work to a team. 282 items (131 open / 151 sealed)
  • Everyday language — short everyday messages: intents, assistant requests, reading a detail out of a text. 115 items (98 open / 17 sealed)
  • Safety & security — untrusted or injected instructions, fraud, moderation, access and security triage. 128 items (40 open / 88 sealed)

Use cases (TypeSafe categories)

Super Decider: No per-category values are published for this system.

Radar: use cases (typesafe categories), two systemsUse cases (TypeSafe categories), Jev 1.13.0 vs Super Decider. Search & retrieval: 95.9 vs —; Model routing: 96.5 vs —; LLM guardrails: 67.0 vs —; Lead generation: 88.1 vs —; Customer support: 91.9 vs —; Insurance claims: 67.5 vs —; Financial crime: 58.3 vs —; Legal & compliance: 74.0 vs —; E-commerce: 89.8 vs —; Risk assessment: 80.1 vs —. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Search & retrieval: Jev 1.13.0: 95.9; Super Decider: No measured result — Search and retrieval: scoring query-to-candidate relevance, reranking results, selecting useful context for RAG. 54 items (18 open, 36 sealed).Search &retrieval95.9 · n/aModel routing: Jev 1.13.0: 96.5; Super Decider: No measured result — Model routing: deciding which LLM or model tier should handle a prompt; classifying a request's intent, domain, difficulty or risk to pick a model. 132 items (131 open, 1 sealed).Modelrouting96.5 · n/aLLM guardrails: Jev 1.13.0: 67.0; Super Decider: No measured result — LLM guardrails: checking LLM inputs, outputs and tool calls: prompt injection, jailbreaks, policy violations, tool-call errors, answer quality. 339 items (209 open, 130 sealed).LLMguardrails67.0 · n/aLead generation: Jev 1.13.0: 88.1; Super Decider: No measured result — Lead generation: matching companies or inbound messages to an ideal customer profile, buyer intent, prioritising and routing sales leads. 98 items (61 open, 37 sealed).Leadgeneration88.1 · n/aCustomer support: Jev 1.13.0: 91.9; Super Decider: No measured result — Customer support: classifying tickets and customer intents, urgency, refunds, routing cases to a team, checking support replies. 389 items (176 open, 213 sealed).Customersupport91.9 · n/aInsurance claims: Jev 1.13.0: 67.5; Super Decider: No measured result — Insurance claims: first-notice-of-loss reports, claim complexity, missing information, claim triage. 31 items (14 open, 17 sealed).Insuranceclaims67.5 · n/aFinancial crime: Jev 1.13.0: 58.3; Super Decider: No measured result — Financial crime: suspicious transactions, KYC, fraud alerts, entity matching for investigations. 31 items (10 open, 21 sealed).Financialcrime58.3 · n/aLegal & compliance: Jev 1.13.0: 74.0; Super Decider: No measured result — Legal and compliance: contracts, policies, regulations, eligibility rules, checking documents against written requirements. 508 items (261 open, 247 sealed).Legal &compliance74.0 · n/aE-commerce: Jev 1.13.0: 89.8; Super Decider: No measured result — E-commerce marketplaces: product listings, product attributes, orders and deliveries, seller catalog policy. 71 items (48 open, 23 sealed).E-commerce89.8 · n/aRisk assessment: Jev 1.13.0: 80.1; Super Decider: No measured result — Risk assessment: estimating probabilities and severity of risks, incidents, vendor or underwriting risk. 78 items (39 open, 39 sealed).Riskassessment80.1 · n/a

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

The real-world application area of each decision, after the TypeSafe use-case map; an item can count in two. Chance-corrected competence per category (0 = chance, below chance negative, 100 = perfect), Jev 1.13.0: open+sealed; Super Decider: open+sealed items pooled; only categories with at least 30 items are spokes, smaller ones are listed below. Hover a category for its definition and item count.

Low sample, n < 30 — indicative only

Category (items)A: Jev 1.13.0B: Super Decider
Graphs and knowledge graphs (15)100.0 n=15—

No published value for either system (under 15 answered items, or no per-category values): Scientific discovery (1), Semantic code linting (4), Feature extraction for predictive modeling (7), Recruiting (3), Moderation and trust and safety (12), Advertising (2).

Not drawn on the radar: with so few items a single answer moves a category score by several points, so these values are noise-prone. These values become spokes once the item pool reaches 30 items per category.

What each category means · items per category
  • Search and retrieval — scoring query-to-candidate relevance, reranking results, selecting useful context for RAG. 54 items (18 open / 36 sealed)
  • Model routing — deciding which LLM or model tier should handle a prompt; classifying a request's intent, domain, difficulty or risk to pick a model. 132 items (131 open / 1 sealed)
  • LLM guardrails — checking LLM inputs, outputs and tool calls: prompt injection, jailbreaks, policy violations, tool-call errors, answer quality. 339 items (209 open / 130 sealed)
  • Lead generation — matching companies or inbound messages to an ideal customer profile, buyer intent, prioritising and routing sales leads. 98 items (61 open / 37 sealed)
  • Customer support — classifying tickets and customer intents, urgency, refunds, routing cases to a team, checking support replies. 389 items (176 open / 213 sealed)
  • Insurance claims — first-notice-of-loss reports, claim complexity, missing information, claim triage. 31 items (14 open / 17 sealed)
  • Financial crime — suspicious transactions, KYC, fraud alerts, entity matching for investigations. 31 items (10 open / 21 sealed)
  • Legal and compliance — contracts, policies, regulations, eligibility rules, checking documents against written requirements. 508 items (261 open / 247 sealed)
  • E-commerce marketplaces — product listings, product attributes, orders and deliveries, seller catalog policy. 71 items (48 open / 23 sealed)
  • Risk assessment — estimating probabilities and severity of risks, incidents, vendor or underwriting risk. 78 items (39 open / 39 sealed)

Not drawn: Other — none of the above: general reasoning, arithmetic or date puzzles, everyday assistant requests without a business workflow. 159 items (76 open / 83 sealed) — not a use case of its own, so it is counted but not drawn.

No items in this release: Gaming, Demand forecasting.

Competence per request type, open / sealed

Radar: competence per request type, open / sealed, two systemsCompetence per request type, open / sealed, Jev 1.13.0 vs Super Decider. Choice · open: 79.0 vs 82.9; Choice · sealed: 75.8 vs 81.3; Noul · open: 54.5 vs 84.0; Noul · sealed: 46.1 vs 84.0; Score · open: 63.3 vs 69.5; Score · sealed: 63.0 vs 71.6. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Choice · open: Jev 1.13.0: 79.0; Super Decider: 82.9Choice · open79.0 · 82.9Choice · sealed: Jev 1.13.0: 75.8; Super Decider: 81.3Choice ·sealed75.8 · 81.3Noul · open: Jev 1.13.0: 54.5; Super Decider: 84.0Noul · open54.5 · 84.0Noul · sealed: Jev 1.13.0: 46.1; Super Decider: 84.0Noul · sealed46.1 · 84.0Score · open: Jev 1.13.0: 63.3; Super Decider: 69.5Score · open63.3 · 69.5Score · sealed: Jev 1.13.0: 63.0; Super Decider: 71.6Score ·sealed63.0 · 71.6

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Chance-corrected competence for Choice, Noul and Score on the 300 open and 1200 sealed decisions.

Competence per tier — open set

Radar: competence per tier — open set, two systemsCompetence per tier — open set, Jev 1.13.0 vs Super Decider. Easy: 93.9 vs 97.7; Standard: 63.2 vs 77.2; Judge: 67.7 vs 75.0; Hard: 65.6 vs 79.5. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Easy: Jev 1.13.0: 93.9; Super Decider: 97.7Easy93.9 · 97.7Standard: Jev 1.13.0: 63.2; Super Decider: 77.2Standard63.2 · 77.2Judge: Jev 1.13.0: 67.7; Super Decider: 75.0Judge67.7 · 75.0Hard: Jev 1.13.0: 65.6; Super Decider: 79.5Hard65.6 · 79.5

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Per-tier competence, the three request types pooled by their published decision counts.

Competence per tier — sealed set

Radar: competence per tier — sealed set, two systemsCompetence per tier — sealed set, Jev 1.13.0 vs Super Decider. Easy: 78.4 vs 88.6; Standard: 68.7 vs 86.2; Judge: 66.9 vs 80.3; Hard: 58.9 vs 72.7. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Easy: Jev 1.13.0: 78.4; Super Decider: 88.6Easy78.4 · 88.6Standard: Jev 1.13.0: 68.7; Super Decider: 86.2Standard68.7 · 86.2Judge: Jev 1.13.0: 66.9; Super Decider: 80.3Judge66.9 · 80.3Hard: Jev 1.13.0: 58.9; Super Decider: 72.7Hard58.9 · 72.7

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Per-tier competence on the sealed decisions, types pooled the same way; item text stays private.
How the categories were made

Chance-corrected competence (0 = chance, 100 = perfect) over the category's items, open and sealed pooled, all tiers: per request type the method's competence cell (Choice and Noul accuracy above chance, Score 1 - error / chance error), then averaged over types weighted by item count. Not part of the JevBench Score; compare systems within a category, not categories with each other. Values can be negative (below chance).

Every one of the 1,624 decisions was labelled with one subject topic (the v1.2 topic list, unchanged) and one or two TypeSafe use-case categories (docs.typesafe.ai/concepts/use-case-map, plus Other) by Winnow-12B Q8 (the model behind System1 Models s1-pro) on our own GPU pod; 1,118 of the 3,248 calls were first made on s1-pro itself and agree 98.2 % with the pod run. Sealed items stayed on our own infrastructure. About 5 % of public items were checked by hand (Claude Opus 5.5); the item-group rules below fix the systematic misses found.

  • routing, routing_hard and tool_selection items (choose the model, agent or tool that handles a request) are Model routing first; the labeller tagged many by the request subject.
  • tool_guardrail, unsafe_action, adequacy, judge_hard and policy_compliance items (check an agent tool call or a drafted answer before it goes out) are LLM guardrails first.
  • lead_qualification items take the topic Finance & commerce (sales); the labeller read the scoring rubric as rules & law.
  • A second use case is kept when the labeller gave it probability >= 0.15 and it is not Other.
All values as a table
SpokeA: Jev 1.13.0B: Super Decider
The four score axes
Intelligence63.678.9
Calibration90.690.8
Speed91.285.3
Cost54.764.3
Capability by subject topic
Math & numbers40.5—
Coding & software80.3—
Rules, policy & law73.7—
Finance & commerce83.4—
Support & operations91.5—
Everyday language96.1—
Safety & security67.7—
Use cases (TypeSafe categories)
Search & retrieval95.9—
Model routing96.5—
LLM guardrails67.0—
Lead generation88.1—
Customer support91.9—
Insurance claims67.5—
Financial crime58.3—
Legal & compliance74.0—
E-commerce89.8—
Risk assessment80.1—
Competence per request type, open / sealed
Choice · open79.082.9
Choice · sealed75.881.3
Noul · open54.584.0
Noul · sealed46.184.0
Score · open63.369.5
Score · sealed63.071.6
Competence per tier — open set
Easy93.997.7
Standard63.277.2
Judge67.775.0
Hard65.679.5
Competence per tier — sealed set
Easy78.488.6
Standard68.786.2
Judge66.980.3
Hard58.972.7

Category radars count each answered item once from the pools named under each radar. API overlay rows use A4+P+L1+L2+L3; A5+P+L1+L2+L3; S+P+L1+L2+L3. Raw and unequated; cells under 15 answered items are omitted. Per-type and tier radars retain each row's original measurement pools: A4/A5 rows have 300 open plus 300 sealed items; full-set rows have S 1,200 plus P 300.

Languages

Raw chance-corrected competence per item language (0 = chance, 100 = perfect; can be negative), from each system's own measured items: Self-hosted rows use S 1,200 + P 300 + L1 (354 items) + L2 (33 items) + L3 (1,118 items), where answered. API rows re-run on A4/A5 use their 300/300-item sealed subset instead of S. Every row’s tag lists the pools it actually answered. L3 is a sealed language supplement (1,118 items), drawn 2026-10-07. Every language has at least 60 items in P ∪ L1 ∪ L2 ∪ L3. Header counts show the full S + P + L1 + L2 + L3 pool. Headline scores are unchanged; these raw cells are unequated, scored for language/category views only and outside the Composite. L3 items were written natively by Claude Sonnet 5.5, each solved blind and language-checked by GPT-6.1 Sol, with gold kept only when both agree or a second review confirms; no gold comes from Jev or any measured API. L3 is API-facing by design and is excluded from future headline draws. Rows with unfinished runs retain their actual coverage tags. L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of OpenAI rows (OpenAI Decisions, GPT-6 Luna, GPT-5.6 Luna) carry that exposure; headline scores do not use L3. C1 adds English items for thin radar categories (everyday language, safety and the other use case). L4 adds 20 new Hindi items for Fastino GLiNER 2.5 Decide only. They were authored with OpenAI and independently solved and language-reviewed with Anthropic before sealing. Seventeen L4 requests succeeded; three HTTP 400 failures remain in the scored observations and do not count as completed coverage. L4 is outside headline and category scores and the common header counts. L5 adds 60 new items for OpenJev DeBERTa v3 Large only: 20 each in Arabic, Hindi and Greek. They were authored with OpenAI and independently solved and language-reviewed with Anthropic before sealing, then checked against the original 512-token native context before keeper selection. All 60 native requests succeeded. Previous failures remain scored and do not count as completed coverage. L5 is outside headline and category scores and the common header counts. Cells under 15 completed supported responses are left empty; recorded input refusals count toward coverage. A dagger (†) marks fewer than 30 completed responses. Competence retains all supported scored observations, including failures and refusals. The mixed-language group is listed first, then English (1,306 items); the other 21 languages and the mixed-language group share 1699 items.

305 systems

JevBench v1.7.2 competence by item language and system
781306959088848383808077757575737372727171706965
Super DeciderAPI · not measured—818777†7686—76†—80†———————72†81†———66†
kettei-fatAPI · S+P (draw v1.6-regular-20261012)—809076†8687—81†—81†———————79†86†———78†
Fastino GLiDE (hosted API, 22 Oct 2026)API · S+P (draw v1.6-regular-20261012)—767478†7993—66†—47†———————70†49†———94†
Torchcast Decision 4B (v1.3)self-hosted · S+P (draw v1.6-regular-20261012)—667275†7168—82†—62†———————70†27†———66†
d1-ultraAPI · S+P (draw v1.6-regular-20261012)—706686†6789—63†—24†———————69†28†———44†
SubQ Snap 1.1API · S+P (draw v1.6-regular-20261012)—757576†8391—61†—39†———————47†77†———78†
Drex MaxAPI · S+P (draw v1.6-regular-20261012)—797285†8288—76†—54†———————71†70†———94†
Bespoke Nimble-v4-maxAPI · S+P (draw v1.6-regular-20261012)—717065†7382—71†—27†———————34†59†———61†
Super Decider (thinking)API · not measured—828180†6788—74†—78†———————68†81†———83†
SubQ SnapAPI · S+P (draw v1.6-regular-20261012)—746862†7284—67†—39†———————51†61†———66†
H2O-Lightning-4B v1.1self-hosted · S+P+L1+L2+L38563555960544938545841373446264946394855473123
Sage 1.3.0API · S+P+L1+L2+L38071485542494445575136444332264451656250593743
kettei-chubAPI · S+P (draw v1.6-regular-20261012)—627763†6183—77†—49†———————24†28†———72†
H2O-Lightning-31B v1.0.7self-hosted · S+P (draw v1.6-regular-20261012)—816675†8391—78†—65†———————75†28†———77†
Torchcast Decision 4B (v1.2)self-hosted · S+P (draw v1.6-regular-20261012)—606568†7678—71†—63†———————39†9†———74†
d1API · S+P+L1+L2+L38066325534443336644028304128143327485141471232
Mercury DecideAPI · not measured———————————————————————
decisio v0.8.0 on gemma-4-31B-itself-hosted · not measured———————————————————————
basal-omni-1.0-ultraself-hosted · S+P (draw v1.6-regular-20261012)—777670†6586—62†—58†———————71†39†———78†
Jev 1.13.0API · S+P+L1+L2+L3766838442248353436363015431173420364323471921
Microsoft-Decision-1 (Azure Foundry) — A2 re-runAPI · A2+P (draw v1.6-regular-20261012)—59——53†——————————————————
Quyet-1.0-Largeself-hosted · S+P+L1+L2+L38476475756665440665543525943296459676746554564
Celeris-1 DecisionAPI · S+P (draw v1.6-regular-20261012)—595951†5688—72†—43†———————52†11†———55†
Torchcast Decision 4B (v1.1)self-hosted · S+P (draw v1.6-regular-20261012)—546265†7261—80†—46†———————33†0†———59†
Myles-G 31BAPI · S+P (draw v1.6-regular-20261012)—757171†6889—59†—40†———————76†38†———67†
Arcbright Elf 1 31Bself-hosted · not measured—726771†7486—59†—50†———————73†24†———77†
decider-12b v2self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8765515861404955614954435546435759675856405836
Xevn-4Bself-hosted · not measured—496372†4472—77†—56†———————37†25†———70†
Jev 1.14.0API · S+P (draw v1.6-regular-20261012)—637373†4275—51†—0†———————39†0†———44†
torchcast-decision-12bself-hosted · S+P+L1+L2+L38665496447483953645044474544335440685354445844
decider-12b v1, stock Gemma-4-12B-itself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8562485049424654615142464738324944615049405143
metask-jev-rain-31B-trainedself-hosted · S+P (draw v1.6-regular-20261012)—716968†6390—57†—46†———————73†36†———78†
vLLM-SR d3-miniself-hosted · not measured—526257†5673—80†—36†———————30†0†———28†
Bobcat 1.3self-hosted · not measured—697170†6387—62†—46†———————74†37†———66†
Microsoft-Decision-1API · not measured———————————————————————
wity-1 (Wity, reasoning always)API · S+P+L1+L2+L3757357683250334446432940173632294144554331185
wity-1 (Wity, reasoning auto)API · S+P+L1+L2+L38373596331602940333844333954241642563936451516
Chon-1.0-12B-Fastself-hosted · not measured—567175†7888—49†—49†———————71†31†———44†
Winnow-12B Q8self-hosted · S+P+L1+L2+L37864405939523954534541294433414636575534384440
deck-31Bself-hosted · S+P+L1+L2+L39178606955686560747156587150536860706159554560
Jeff-1.0-Largeself-hosted · S+P (draw v1.6-fastlane-20261009)99†718967†649156†—98†49†——————77†84†62†———59†
Cygnetself-hosted · S+P+L1+L2+L37957385641413239634534274236214245535434394640
Myles-G 12BAPI · S+P (draw v1.6-regular-20261012)—566673†5986—58†—48†———————82†4†———43†
Wald 26B-A4Bself-hosted · not measured—646568†7472—60†—36†———————58†29†———78†
decisio v0.8.0 on gemma-4-12B-itself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7956395539423338634129253435253744485638404227
attest-31B-v1.0API · S+P (draw v1.6-regular-20261012)—676774†5889—53†—45†———————82†26†———67†
Sage 1.4API · S+P (draw v1.6-regular-20261012)—656573†6583—67†—39†———————34†52†———61†
decisio v0.9.0 on gemma-4-12B-itself-hosted · S+P+L1+L2+L38156415538403141624128293634253744485637404331
Jev-Omniself-hosted · S+P+L1+L2+L38656385859444537585038274928255038575547484637
metask-jev-rain-12B-trainedself-hosted · S+P (draw v1.6-regular-20261012)—647481†7285—49†—45†———————59†25†———77†
Bobcat Flash 1.2self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8271454738543333514143373928264737485044582927
Xor 26B-A4Bself-hosted · S+P+L1+L2+L38061464048565136524434454332335040685050504246
SPX-CD FlashAPI · S+P+L1+L2+L3786338433640273449483122301783824375146501229
Messier One v0.4self-hosted · S+P (draw v1.6-regular-20261012)—516862†7072—65†—39†———————32†18†———55†
decider chat on Gemma-4-31B-itself-hosted · S+P+L1+L2+L37564585434535045565945395245324344514443444449
Hopper 12B trainedself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7954505142483339544232233828163632484734322434
metask-jev-rain-31B-trained service v3self-hosted · S+P (draw v1.6-regular-20261012)—746874†6382—55†—65†———————80†43†———78†
Surogate Rune 26B-A4B v3 (v1.6 pool)self-hosted · S+P+L1+L2+L37559474336473737454433374538195237554340502333
GEV-26B-Decideself-hosted · S+P+L1+L2+L37852333731423742372831233028213529422140282725
SPX-CD-Omniself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8053354046362932504441344426314822445635382434
Metask rain 4Bself-hosted · S+P (draw v1.6-fastlane-20261009)100†486574†606261†—30†8†——————0†41†0†———73†
shisa-de-1self-hosted · S+P (draw v1.6-regular-20261012)—456176†4588—52†—30†———————52†25†———50†
OpenAI DecisionsAPI · A5+P+L1+L2+L3L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of this row carry that exposure; headline scores do not use L3.7659364841552737433724234637303327403339374025
InstinctAPI · A4+P+L1+L2+L3675822353228273152482018341923223939394243137
Blink v0.3 26B-A4Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—4654†90†457054†77†39†25———————39————0†
HS1 0.1.2API · S+P (draw v1.6-regular-20261012)—505473†4678—61†—35†———————58†37†———39†
baselabs elephant-1self-hosted · not measured—537472†6272—54†—32†———————70†24†———55†
Diffusion Jevself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8259515749653741524448294345364944635050513816
René-1 31B FP8self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7567453942464031553735375413254433474342462043
Amazon Strands Decider 26B-A4Bself-hosted · not measured—436849†4969—59†—56†———————32†15†———40†
Wald 4B v2.1self-hosted · S+P (draw v1.6-regular-20261010-a2)—5072†75†345543†54†54†49———————25————12†
pplx-decider-flash-v1.2self-hosted · S+P (draw v1.6-regular-20261012)—476665†5474—86†—24†———————0†0†———22†
Bespoke Nimble 9B v3self-hosted · S+P+L1+L2+L3806145503544303056412774032173623324260542037
Aplomb 1self-hosted · S+P+L1+L2+L3The headline keeps the original published measurement. Its originally pinned Hugging Face revision (f8e2dc81) was later deleted upstream, so the language and category supplements (L1, L2, L3) were measured on 12 Oct 2026 with the current upstream revision d6743291 (all file hashes recorded) on an isolated pod. That revision reproduces the original answers: 98 % agreement on the 300 public items (published bar 95 %) and 97.5 % on all 1,500 headline items. Each supplement carries the original P300 answers by stable item identity; they count once. All new (sealed) supplement items were answered; the only failed requests (3 public items, GPU out-of-memory) are covered by the original P300 answers. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.694643362641212244272214242811272725433338132
Respan Span-02self-hosted · S+P (draw v1.6-regular-20261012)—605057†6180—53†—37†———————70†29†———72†
Seb-9B-v2self-hosted · not measured—446574†3168—86†—33†———————19†5†———22†
wity-1 (Wity, reasoning off)API · S+P+L1+L2+L367523848293621261013171417176112430313333261
APUS-OpenJev-v1-9Bself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7448333125371822374180202561628293034351712
Arcbright Elf 1 12Bself-hosted · S+P (draw v1.6-regular-20261012)—415360†5779—65†—30†———————42†0†———33†
Jeffan-31b-1710self-hosted · not measured—767158†6777—69†—45†———————76†58†———67†
Plumb-4Bself-hosted · S+P+L1+L2+L3724535453630382732342583240142431252827392220
SPX-CD ProAPI · S+P+L1+L2+L38270445537445343625035474119224737485429522947
Chon-1.0-E4Bself-hosted · not measured———————————————————————
Messier One v0.2self-hosted · not measured———————————————————————
spark-s1-4b-v6self-hosted · S+P+L1+L2+L380463331182891737352202223015292023102627
KnowLine 31B-Gen1self-hosted · not measured—817268†7588—63†—46†———————80†53†———67†
swanOneself-hosted · S+PThe pinned model weights cannot currently be accessed (Hugging Face returned HTTP 401). Supplement measurements remain pending access recovery; the radar shows the existing S + P items.—578668†4885†26†45†91†42†—————————————
AJev lm6qself-hosted · not measured—746876†6784—64†—43†———————77†59†———65†
Quyet-1.0-Mediumself-hosted · S+P+L1+L2+L360443332254225153532238152002933351921351214
Bespoke Nimble-v4API · S+P (draw v1.6-regular-20261012)—426751†3973—72†—15†———————22†0†———33†
Turion Jev 31Bself-hosted · not measured—757250†6283—68†—54†———————71†61†———72†
Vansa-3.4API · A4+P+L1+L2+L35447124228171210244311013191291726153335011
JPT-4Bself-hosted · S+P (draw v1.6-regular-20261012)—367563†2775—70†—30†———————11†14†———33†
levself-hosted · S+P+L1+L2+L35541282022221222282229822112141728192437537
NInfer Qwen3.8-Flash-Next mixedself-hosted · S+P+L1+L2+L37253415024333033553328164320273635543138432520
decider-4b v2self-hosted · S+P+L1+L2+L36242412511288222132320181318183222242911
baselabs mammoth-1self-hosted · not measured—777683†6590—65†—62†———————79†76†———61†
Jeffan-4B-2010self-hosted · not measured—396676†3063—83†—14†———————20†0†———22†
jev-localself-hosted · S+P+L1+L2+L37146433121312122333217720127212910342529115
GPT-6 Luna (low reasoning effort)API · A4+P+L1+L2+L3L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of this row carry that exposure; headline scores do not use L3.9796919391941009410099989694939694969710098969997
Autoloops – Gemma 4 31B ITAPI · A4+P+L1+L2+L37367445044566142616043484848476050554945534151
Wald 4Bself-hosted · S+P (draw v1.6-fastlane-20261009)95†435152†355451†—47†0†——————8†26†14†———15†
GPT-6 Luna (default medium reasoning effort)API · A4+P+L1+L2+L3L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of this row carry that exposure; headline scores do not use L3.9797989691981009999969810096959895979995961009996
ClassOne Qwen 3.5 9Bself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.714538512636231930432301781411301718401331
JevK5 v0.3self-hosted · S+P+L1+L2+L36743404229192013283018028278162817223429318
metask-jev-4bself-hosted · S+P+L1+L2+L37340303415211619282412915409112316122640
deck-4B v1.0self-hosted · S+P+L1+L2+L367433745322221192930220312651323182736291917
Kahn1 4Bself-hosted · S+P+L1+L2+L3674036462835253029452553525211433192841302020
Clef-Flashself-hosted · S+P+L1+L2+L3694440318349194129231810201112232323361311
Decision-4Bself-hosted · S+P+L1+L2+L36143271903107202226261000211713152500
decisor-4bself-hosted · S+P (draw v1.6-fastlane-20261009)100†367053†445824†—45†0†——————50†12†11†———13†
janus 4Bself-hosted · S+P+L1+L2+L36340352315199152120162259001129171520107
Qwen3.5-9B Jev-like data-mix v2self-hosted · S+P+L1+L2+L37146292418376133531120253091830382424321
Seb-9Bself-hosted · not measured———————————————————————
JevK5 v0.2.0self-hosted · S+P+L1+L2+L365362636212123152420601013742421191826132
Tern-35B-A3Bself-hosted · not measured—436758†3676—56†—13†———————33†18†———21†
Amos 1 mediumAPI · S+P (draw v1.6-regular-20261012)—356271†4686—59†—40†———————15†0†———15†
Aleria Judge 1.1.1self-hosted · not measured—706769†6585—58†—49†———————68†28†———53†
JevOneself-hosted · S+P+L1+L2+L37349373522331021323122102010920243040183210
Fastino GLiDEAPI · A4+P+L1+L2+L36973405329424337715243334322333756534745433540
Jebadiah 9B v2.1self-hosted · S+P (draw v1.6-regular-20261012)—356745†3270—62†—25†———————3†5†———22†
Verdin-4Bself-hosted · S+P (draw v1.6-regular-20261012)—315651†4054—69†—52†———————34†5†———21†
TypeCastLM 1.4.0self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.57382425822161728181201790517961519133
blink-4bself-hosted · S+P (draw v1.6-regular-20261012)—317351†3179—66†—33†———————2†14†———22†
jiwo 4bself-hosted · S+P (draw v1.6-regular-20261010-a2)—3053†78†344341†65†20†12———————12————0†
Imajev-4Bself-hosted · S+P+L1+L2+L351373027924019121014090008101352910
SimpleJev Qwen3.6-35B-A3BAPI · A4+P+L1+L2+L36563374226433041535424344417222134302034422520
Uniphore-4B-Deciderself-hosted · S+P (draw v1.6-regular-20261012)—327152†2570—74†—29†———————0†18†———15†
Decision 2.0 Lux 9Bself-hosted · S+P (draw v1.6-regular-20261012)—316467†4067—75†—14†———————19†9†———33†
Hopperself-hosted · S+P+L1+L2+L35634312215220151612130141230121517151100
Malkuth-4Bself-hosted · S+P+L1+L2+L3493721173117177540700000422000
jqvself-hosted · S+P+L1+L2+L3703916241829817151721010501872716252903
JPT-35B-A3Bself-hosted · S+P+L1+L2+L3865639483245342843473228441763934284633451720
decider-35b-a3bself-hosted · S+P+L1+L2+L377523129164523233724291644952422413624381428
CoCo-Decision-4Bself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.5136262321171822281716014902724162025121
Manchego v2.1self-hosted · S+P+L1+L2+L3503325110150191281702080212196473
Decision 4B v1.2self-hosted · S+P+L1+L2+L3653636321625162227342102432222728203534251214
Standard One 3B SH v4self-hosted · not measured—324763†2769—61†—24†———————26†0†———20†
Fastino GLiDE 1.2self-hosted · not measured—777478†7991—67†—52†———————70†49†———94†
Raw Qwen3 4B Instruct 2507 direct logitsself-hosted · S+P+L1+L2+L359341731532017244170103192142519213104
StartLux-Decision v1.1 27B Instantself-hosted · S+P (draw v1.6-regular-20261012)—727886†7686—78†—51†———————82†61†———78†
Standard One 3B SHself-hosted · S+P (draw v1.6-regular-20261012)—287270†2077—68†—22†———————20†0†———21†
RSI-Jev v6.1-VL 27Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—5783†83†517244†90†42†32———————53————0†
GPT-5.6 LunaAPI · A4+P+L1+L2+L3L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of this row carry that exposure; headline scores do not use L3.9692869284869290869490799397948987848591878290
JEV-27Bself-hosted · S+P+L1+L2+L3716236422341363754393337439194737454737432648
torchcast-decision-27bself-hosted · S+P+L1+L2+L38174495851454340675242534723215038576139513544
Bespoke Nimble 9Bself-hosted · S+P+L1+L2+L36645413635411021372619423912625263940321411
Nimble-v4-maxself-hosted · not measured—746965†7983—72†—31†———————52†69†———55†
pplx-decider-v1.2self-hosted · S+P (draw v1.6-regular-20261012)—737877†7887—73†—35†———————43†48†———61†
Perplexity Decider v1.1 27Bself-hosted · S+P+L1+L2+L38579505862634352705148425721386333676442643957
vLLM-SR d3-turboself-hosted · S+P (draw v1.6-regular-20261012)—757479†7683—78†—37†———————57†78†———72†
vLLM-SR d3self-hosted · S+P (draw v1.6-regular-20261012)—747379†7986—73†—39†———————51†69†———55†
reflex 4Bself-hosted · S+P+L1+L2+L356333324413113151211061117017172941804
Vera Largeself-hosted · not measured—697381†5880—88†—38†———————49†45†———55†
Amos 1 largeAPI · S+P (draw v1.6-regular-20261012)—707164†7784—68†—44†———————62†62†———53†
JADEself-hosted · S+P+L1+L2+L320000000000000000000000
Nagi-ENORMOUS 27Bself-hosted · S+P (draw v1.6-regular-20261012)—626675†7784—58†—22†———————50†23†———39†
typecastlmself-hosted · S+P+L1+L2+L3542930196246112423130571081514820133
Jebadiah 27Bself-hosted · S+P+L1+L2+L36564364730342239514129183718113133494629441726
iota-decide-4bself-hosted · S+P (draw v1.6-regular-20261012)—247032†1668—64†—15†———————6†0†———16†
Decision 2.0 Vega 27Bself-hosted · S+P+L1+L2+L36864394923362630353641304418222930465121432140
Jebadiah 9B v2self-hosted · S+P (draw v1.6-regular-20261012)—276742†3167—65†—0†———————6†18†———24†
Standard One 8B SHself-hosted · S+P (draw v1.6-regular-20261010-a2)—3044†77†414523†43†0†9———————4————0†
Decision 4B v1.1self-hosted · S+P+L1+L2+L3593329391224102124411701519111822727282255
AutoJev-27B (denis-pplx, Qwen3.8-27B)self-hosted · S+P+L1+L2+L37459413727353332513830244216182736424832452527
pjev-2bself-hosted · S+P (draw v1.6-regular-20261012)—255348†3552—57†—27†———————9†23†———23†
Eikos-27Bself-hosted · S+P+L1+L2+L37568455440465644505838414626304139515343504249
Instinct Dual 4BAPI · A4+P+L1+L2+L3363212500027132000001161201500
NInfer Qwen3.8-27B NVFP4self-hosted · S+P+L1+L2+L3725545513540322341423322382217183758372643178
Compass 0.2.0self-hosted · S+P (draw v1.6-regular-20261012)—104525†1634—22†—17†———————0†0†———36†
JEB-35B-A3Bself-hosted · S+P (draw v1.6-regular-20261012)—294942†4064—64†—17†———————30†7†———24†
OpenJev (thinking, BF16)self-hosted · S+P+L1+L2+L38778595431553160646653506543536046536251564142
Open-Jev 9Bself-hosted · S+P+L1+L2+L370474048234426212635180245142819351530381417
Gemini 3.1 Flash-LiteAPI · A4+P+L1+L2+L38468426457535846635842455551274340493554544944
Standard One 8Bself-hosted · S+P+L1+L2+L362382217201712426197012304023142334610
Clefself-hosted · S+P+L1+L2+L38265504927493846554533434321303136524437492336
ArseneLupin v1.1self-hosted · S+P (draw v1.6-regular-20261012)—166122†2040—44†—7†———————21†14†———5†
system-one-openAPI · A4+P+L1+L2+L3233181491113167166000001910022502
JEV Qwen3.5-9B Base NVFP4self-hosted · S+P+L1+L2+L34027241261611211700700025130705
Arbiter-4Bself-hosted · S+P (draw v1.6-regular-20261012)—275950†2066—76†—21†———————0†0†———24†
Decider-4Bself-hosted · S+P (draw v1.6-regular-20261012)—216224†1354—87†—25†———————4†4†———32†
ClassOne Gemma 4 E4Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—2258†50†502125†57†31†0———————34————0†
SemIf, formerly OpenJevself-hosted · S+P+L1+L2+L3583015162225101415901900011157112000
openjev-sglangAPI · A4+P+L1+L2+L36047202919280192442191212139133224143728200
DecisionNode Edge 0.9.2self-hosted · not measured—204766†4869—41†—33†———————12†0†———16†
RYOTIDE-Qwen9self-hosted · S+P (draw v1.6-fastlane-20261009)92†197045†296015†—13†0†——————0†1†15†———28†
ClassOne Qwen 3.5 4Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—2155†60†312337†59†27†1———————9————0†
Jez-large-1.0.0self-hosted · not measured—94823†2362—61†—12†———————1†0†———8†
RYOTIDE-Qwenself-hosted · S+P (draw v1.6-regular-20261012)—227517†3373—77†—25†———————33†33†———26†
Chon-1.0-12Bself-hosted · S+P (draw v1.6-regular-20261012)—63429†6250—24†—0†———————28†0†———44†
Nomaself-hosted · S+P (draw v1.6-regular-20261012)—205722†1931—79†—25†———————1†0†———44†
KnowLine 35B-A3B-Gen1.5self-hosted · not measured—617159†5092—75†—15†———————42†44†———44†
Bev / Bonsai 27Bself-hosted · S+P+L1+L2+L367462635193014172522160900024286212746
LitJevself-hosted · S+P+L1+L2+L369493439273630233434280242210103239402930131
local-jev Qwen3.5-4Bself-hosted · S+P+L1+L2+L33515111105028030000078001100
system-oneself-hosted · S+P+L1+L2+L3513162743671719600700101618011300
StartLux-Decision v1.1 27B AutoThinkself-hosted · S+P (draw v1.6-regular-20261012)—667187†6184—75†—54†———————68†61†———72†
kev 8Bself-hosted · S+P+L1+L2+L351352426922087070040002092700
Decision 2Bself-hosted · S+P+L1+L2+L320147001000000000003000000
OpenSourceJevself-hosted · S+P+L1+L2+L33320185211050000000000001000
Imajev-9Bself-hosted · S+P (draw v1.6-regular-20261012)—336349†4065—64†—29†———————9†0†———15†
open-alternative-jevself-hosted · S+P+L1+L2+L34324203014373911003007274401
Raw Qwen3 8B direct logitsself-hosted · S+P+L1+L2+L3543071503001117718816000122751400
Liquid AI d1-3Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—1434†55†3060†41†29†0———————1————0†
Krino-2Bself-hosted · S+P (draw v1.6-regular-20261012)—161221†4345—19†—22†———————0†0†———12†
Imajev-2Bself-hosted · S+P (draw v1.6-regular-20261012)—54720†1742—0†—34†———————0†0†———19†
decider-2bself-hosted · S+P+L1+L2+L34225770142432802300581200700
Nemotron Diffusion 8Bself-hosted · S+P+L1+L2+L330160016190060004012001800300
Nev-2Bself-hosted · not measured———————————————————————
DeepSeek V4.1 FlashAPI · A4+P+L1+L2+L39385847137836597687279658680785979859082876257
kev 4Bself-hosted · S+P+L1+L2+L344252811014051907016000013001600
Clef-omniself-hosted · S+P (draw v1.6-regular-20261010-a2)—3554†74†365224†51†17†18———————45————0†
Kev 27Bself-hosted · S+P+L1+L2+L37769475038424244644441454420265031515535482233
FrontiersMind Lumma-fev-27Bself-hosted · not measured—556469†5176—62†—26†———————33†2†———55†
DecisionNode Edge Mini 0.9.2self-hosted · not measured—33830†4546—41†—19†———————0†3†———22†
jiwo 0.8bself-hosted · S+P (draw v1.6-regular-20261010-a2)—034†59†2570†55†0†0———————0————0†
Malkuth-2Bself-hosted · S+P+L1+L2+L33514202600000000000000400
Qwen3-Reranker-4Bself-hosted · S+P+L1+L2+L340000300000000000000000
SimpleJev Qwen3.8-27B (self-hosted, v1.6 pool)self-hosted · S+P+L1+L2+L38062445234414329614133243925293241495031502926
ZeroEntropy zerank-2self-hosted · S+P+L1+L2+L300000000000000000000000
EXAONE-4.0-1.2B-JEV v0.3self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.56000000000000000000000
Open-Jev 2Bself-hosted · S+P+L1+L2+L339181116171009712000000003600
SimpleJev Qwen3.8-27BAPI · A4+P+L1+L2+L37063284038433322544323253417312144433731552927
Vega 4Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—824†15†43422†42†10†6———————0————0†
Open-Jev 27B v1.1self-hosted · S+P+L1+L2+L3645544412240242144332837388112311423634372423
Raw Phi-4 mini direct logitsself-hosted · S+P+L1+L2+L313265000010007000010001502
decision-machine-1API · A4+P+L1+L2+L3104000001000050000004005
SimpleJevself-hosted · S+P+L1+L2+L3681100000000000000000960
Jeetu Decider 2Bself-hosted · not measured—060†2023—42†—0†———————0†0†———18†
JevActAPI · A4+P+L1+L2+L312001000000000000001001400
smalljev semantic-v9self-hosted · S+P+L1+L2+L300000000000000000000000
Gutsy 0.8B v0.3self-hosted · S+P (draw v1.6-regular-20261010-a2)—014†27†1220†28†0†0———————0————0†
Vega-U (Qwen3.5-4B reader)self-hosted · S+P (draw v1.6-regular-20261012)—000†1010—0†—3†———————0†0†———0†
PurpleMIST-Mini-1.0self-hosted · S+P (draw v1.6-regular-20261012)—000†00—0†—1†———————0†2†———0†
kev 0.6Bself-hosted · S+P+L1+L2+L310000000000000000000000
jul fastself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.00000000000000000000000
OpenDecisionself-hosted · S+P+L1+L2+L301000000000000000000000
ClassOne Gemma 4 E2Bself-hosted · not measuredThis model's server gives different answers after each restart, so a supplement run cannot reproduce the row's original answers (87.7 % agreement on the 300 public items, below the 95 % bar). Cause (verified 12 Oct 2026): the published model has no trained embeddings for its 12 decision delimiter tokens, so they are randomly initialised at every server start. Its language and category supplements are not published; the radar shows the existing S + P items.———————————————————————
Qwen3.5-0.8B Decision Modelself-hosted · not measured———————————————————————
Fastino GLiNER-2.5-DecideAPI · S+P+L1+L2+L3———————————————————————
Deem 0.8B v1self-hosted · S+P+L1+L2+L350000100000000000000800
Decision Fastself-hosted · S+P+L1+L2+L300000000000000000000000
Quyet-1.0-Small-ENself-hosted · S+P+L1+L2+L300000000000000000000000
ClassOne Qwen 3.5 2Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—30†18†1800†39†0†0———————11————0†
Laya typed-decisionsself-hosted · S+P+L1+L2+L300000000000000000000000
Vega-U (Qwen3.5-0.8B reader)self-hosted · S+P (draw v1.6-regular-20261012)—000†00—6†—7†———————0†0†———0†
WaterSheepself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.00000000000000000000000
openJev Verdict 1.4self-hosted · S+P+L300000000000000000000000
Raw Qwen3 0.6B direct logitsself-hosted · S+P+L1+L2+L30000000010000000130400800
lev-350mself-hosted · S+P+L1+L2+L300000000000000000000000
Raw Qwen3 1.7B direct logitsself-hosted · S+P+L1+L2+L3100060000200000000000000
kev 0.5Bself-hosted · S+P+L1+L2+L300000000000000000000000
Jez-small-1.0.0self-hosted · not measured—000†00—4†—0†———————0†0†———0†
openJev Verdictself-hosted · S+P+L1+L2+L300000000000000000000000
Quyet-1.0-Smallself-hosted · S+P+L1+L2+L300000000000000000000000
Reflex Instinct 0.6Bself-hosted · not measured———————————————————————
watt-flash-0.1self-hosted · S+P+L1+L2+L300000000000000000000000
Quyet-1.0-Tinyself-hosted · S+P+L1+L2+L300000000000000000000100
Phocinae-Largha-150M-v1self-hosted · S+P (draw v1.6-regular-20261012)—000†00—0†—0†———————0†0†———0†
Liquid AI d1-omni-600Mself-hosted · S+P (draw v1.6-regular-20261010-a2)—00†0†000†0†0†0———————0————0†
Vega 0.8Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—00†22†000†5†0†0———————0————0†
open-jev-deberta-v3-largeself-hosted · S+P+L1+L2+L3———————————————————————
modernbert-jev v2self-hosted · not measured———————————————————————
Julia-1self-hosted · S+P (draw v1.6-regular-20261012)—003†022—0†—0†———————0†0†———1†
verdict-smallself-hosted · S+P+L1+L2+L300000000000000000000001
Tacet Sonataself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.20000000000000000000000
Layaself-hosted · S+P+L1+L2+L300000000000000000000000
Mixedbread mxbai-rerank-base-v2self-hosted · S+P+L1+L2+L300000000000000000000000
Laya multilingualself-hosted · S+P+L1+L2+L3000000000000005141230000
Certo v1self-hosted · S+P+L1+L2+L300000000000000000000000
CLM-8Bself-hosted · S+P+L1+L2+L300000000000000000000000
BAAI bge-reranker-v2-m3self-hosted · S+P+L1+L2+L300000000000000000000000
Alibaba GTE Reranker ModernBERT-baseself-hosted · S+P+L1+L2+L300000000000000000000000
Mirrorself-hosted · S+P+L1+L2+L300000000000000000000000
Open Jev JSON Canvasself-hosted · S+P+L1+L2+L37665554247563755605551403446443341504754514953
Qwen3.8 27BAPI · A4+P+L1+L2+L31124000000000000000000000
APUS-OpenJev-v1-35B-A3Bself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8254374047402725414734142333123132373935371710
BB-Qwen3.5-4B-LoRAAPI · not measured———————————————————————
Bongard-miniself-hosted · S+P (draw v1.6-regular-20261012)—43413†2643—43†—0†———————0†0†———20†
jevora-9b-v1self-hosted · S+P (draw v1.6-regular-20261012)—143939†2358—65†—0†———————0†0†———16†
Kev-9B v2self-hosted · S+P (draw v1.6-regular-20261012)—304443†3375—72†—19†———————19†32†———22†
Nagi-HUGE 12Bself-hosted · S+P (draw v1.6-regular-20261012)—537370†5784—72†—12†———————42†7†———67†
Nemotron Deciffusion 8B v2self-hosted · S+P (draw v1.6-regular-20261012)—506867†5460—67†—22†———————27†7†———32†
PurpleMIST-Flash-1.0self-hosted · S+P (draw v1.6-regular-20261012)—355733†4877—57†—27†———————14†0†———16†
RYOTIDE-Gemmaself-hosted · S+P (draw v1.6-regular-20261012)—304360†4765—54†—31†———————8†6†———38†
TDE general large v0.1self-hosted · S+P (draw v1.6-regular-20261012)—000†00—0†—0†———————0†0†———0†
Trio-Spark v1.3API · S+P (draw v1.6-regular-20261012)—294560†1357—44†—0†———————0†0†———22†
AutoJev-27B (RTX PRO 6000)self-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7460423827353332514030224015182837464832452527
Bosun v3.1 0.6Bself-hosted · S81+P299+L1+L2+L3Catalogue entry · measured language cells40000000000000000000700
GLiNER2self-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. About 1,000 issued items were lost to out-of-memory kills and a machine shutdown; they stay counted as spent and are never re-asked, so 17 of 23 languages rest on 50 to 59 answered items instead of the 60-item target. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.110000000140000000000120010
GLiNER2 largeself-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.791000000000970001008800
GLiNER2.5 multiself-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.00000000000000000000400
GLiNER2.5 smallself-hosted · P+L1+L2+L3Catalogue entry · measured language cells00000000000000000000000
jeffself-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.00000000000000000000000
Jobe Qwen3.5-4Bself-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.58301416125491415801900011156132000
mica-v01-4bself-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.5538322232602218264125441392230122100
Needle 3self-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from new runs of the supplements L1, L2 and L3 (all items) on the original pinned closed engine, approved by the benchmark owner, on an isolated machine, plus 281 S+P items: 153 from an earlier partial run and 128 never-asked items chosen by their use-case label ('other') to complete that radar spoke (these items also count in the other cells, so the S+P part is not a random sample); the rest of S+P was not run. The engine abstains on about 30 % of items and returns invalid UTF-8 on about 7 % (mostly non-Latin scripts); both count as unanswered. Its chance-corrected competence is below zero in every cell, which the table shows as 0. Its answers to the same public items differ between machines for about 10 % of items. 22 of 23 languages rest on 19 to 59 answered items instead of the 60-item target; the radars are complete. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.00000000000†00000†00†000†00†
Needle 3, options as toolsself-hosted · S+P+L1+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells combine an earlier partial run on the current pools (175 S+P items) with a new run of the supplements L3 (1,382 of 1,418 items) and L1 (329 of 654) on the original pinned closed engine, approved by the benchmark owner, on an isolated machine; L2 and the rest of S+P were not run before the machine's time limit. The engine abstains on about 9 % of items and returns invalid UTF-8 on about 7 % (mostly non-Latin scripts); both count as unanswered. Its chance-corrected competence is below zero in every cell, which the table shows as 0. Its answers to the same public items differ between machines for about 10 % of items. 18 of 23 languages rest on 24 to 59 answered items instead of the 60-item target; the radars are complete. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.0040000000000000†0000030†
NInfer Qwen3.8-27B NVFP4 (T=1.5)self-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.715039512432292038372715371918183351262538145
OpenJev (DiffusionGemma 26B-A4B NVFP4, razorback16)self-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. This model samples stochastically: on the 300 public items its answers in the supplement passes agree with its S+P pass on 86-89 %; those public items count once, from the S+P pass. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.6054464149462937424437303023273128332735313334
reflex-27bself-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.685634382736322047492920282311183137374030135
Surogate Rune 26B-A4B v3 (RTX PRO 6000)self-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells combine its S+P run on the current pools (6 Oct 2026) with new L1, L2 and L3 runs on the same pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7559474336473737454433374538195237554340502333
Vonself-hosted · S+P+L1+L2+L3Catalogue entry · measured language cellsThis historical row keeps its published v1.5 headline. Its language and category cells come from a new measurement on the current item pools (S+P, L1, L2, L3), run with the original pinned model and serving setup on an isolated machine; each item was sent once and failed requests stay scored as failures. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.00000000000000000000000
Wrappers (listed, never ranked)
classifier.devAPI · A4+P+L1+L2+L3737223353643342833412315359183225444417511434
metask-jev-rain-12Bself-hosted · S+P (draw v1.6-fastlane-20261009)99†647978†558367†—71†42†——————67†65†47†———71†

Intelligence gate and Noul decisiveness

The Composite applies a soft gate below Intelligence 50 (× (I/50)²). Rows under it show their value with a "below gate" label. The decisive rate is the share of valid Noul answers with P(yes) ≥ 0.80 or ≤ 0.20 (for systems that return only a label, every valid label counts as decisive); only decisive answers count as answers. Accuracy among decisive answers is a diagnostic and does not enter any score.

SystemIntelligenceChoiceNoulScoreNoul decisive rateAccuracy among decisive
Super Decider78.9nativenativenative100 %93 %
kettei-fat79.8nativenativenative95 %96 %
Fastino GLiDE76.1nativenativenative86 %97 %
Torchcast Decision 4B67.1nativenativenative100 %86 %
d1-ultra69.7nativenativenative86 %94 %
SubQ Snap 1.175.8nativenativenative89 %95 %
Drex Max78.9nativenativenative93 %94 %
Bespoke Nimble-v4-max68.2nativenativenative78 %98 %
Super Decider80.0nativenativenative100 %93 %
SubQ Snap72.3nativenativenative86 %95 %
H2O-Lightning-4B v1.160.0nativenativenative100 %85 %
Sage 1.3.065.6nativenativenative83 %95 %
kettei-chub62.2nativenativenative87 %94 %
H2O-Lightning-31B v1.0.779.3nativenativenative100 %92 %
Torchcast Decision 4B60.3nativenativenative100 %84 %
d162.1nativenativenative83 %93 %
Mercury Decide65.9nativenativenative90 %90 %
decisio v0.8.0 on gemma-4-31B-it70.4nativenativenative94 %96 %
basal-omni-1.0-ultra74.1nativenativenative92 %94 %
Jev 1.13.063.6nativenativenative79 %98 %
Quyet-1.0-Large73.4nativenativenative91 %93 %
Celeris-1 Decision58.9nativenativenative79 %93 %
Torchcast Decision 4B54.2nativenativenative100 %78 %
Myles-G 31B73.6nativenativenative100 %89 %
Arcbright Elf 1 31B71.9nativenativenative100 %91 %
decider-12b v261.9nativenativenative99 %83 %
Xevn-4B52.1nativenativenative100 %79 %
Jev 1.14.059.9nativenativenative76 %95 %
torchcast-decision-12b60.5nativenativenative96 %86 %
decider-12b v1, stock Gemma-4-12B-it59.6nativenativenative96 %87 %
metask-jev-rain-31B-trained70.7nativenativenative100 %88 %
vLLM-SR d3-mini50.8nativenativenative77 %91 %
Bobcat 1.368.1nativenativenative92 %94 %
Microsoft-Decision-157.2nativenativenative77 %93 %
wity-167.9nativenativenative91 %94 %
Chon-1.0-12B-Fast60.3nativenativenative100 %78 %
Winnow-12B Q859.5nativenativenative86 %92 %
deck-31B73.0nativenativenative96 %94 %
Jeff-1.0-Large71.9nativenativenativenot supported—
Cygnet54.8nativenativenative81 %92 %
Myles-G 12B57.3nativenativenative100 %79 %
Wald 26B-A4B65.5nativenativenative100 %82 %
decisio v0.8.0 on gemma-4-12B-it52.3nativenativenative82 %91 %
attest-31B-v1.065.2nativenativenative97 %88 %
Sage 1.463.7nativenativenative91 %91 %
decisio v0.9.0 on gemma-4-12B-it52.3nativenativenative81 %91 %
Jev-Omni55.5nativenativenative74 %94 %
metask-jev-rain-12B-trained66.5nativenativenative100 %86 %
Bobcat Flash 1.265.6nativenativenative89 %92 %
Xor 26B-A4B57.5nativenativenative81 %94 %
SPX-CD Flash58.4nativenativenative81 %94 %
Messier One v0.452.0nativenativenative100 %78 %
decider chat on Gemma-4-31B-it58.5nativenativenative80 %98 %
Hopper 12B trained51.7nativenativenative79 %93 %
metask-jev-rain-31B-trained service v372.6nativenativenative100 %89 %
Surogate Rune 26B-A4B v356.1nativenativenative73 %97 %
GEV-26B-Decide49.8below gatenativenativenative70 %96 %
SPX-CD-Omni48.8below gatenativenativenative68 %93 %
Metask rain 4B51.1nativenativenativenot supported—
shisa-de-149.9below gatenativenativenative76 %90 %
OpenAI Decisions56.9—————
Instinct47.8below gate—————
Blink v0.3 26B-A4B49.2below gatenativenativenative67 %95 %
HS1 0.1.252.9nativenativenative83 %88 %
baselabs elephant-155.8nativenativenative78 %91 %
Diffusion Jev54.9nativenativenative93 %85 %
René-1 31B FP861.7nativenativenative78 %98 %
Amazon Strands Decider 26B-A4B46.7below gatenativenativenative72 %91 %
Wald 4B v2.152.0nativenativenative100 %78 %
pplx-decider-flash-v1.245.5below gatenativenativenative79 %87 %
Bespoke Nimble 9B v356.5nativenativenative87 %89 %
Aplomb 145.2below gatenativenativenative68 %93 %
Respan Span-0260.9nativenativenative90 %87 %
Seb-9B-v246.1below gatenativenativenative70 %91 %
APUS-OpenJev-v1-9B47.2below gatenativenativenative88 %85 %
Arcbright Elf 1 12B43.8below gatenativenativenative63 %97 %
Jeffan-31b-171071.6nativenativenative87 %94 %
Plumb-4B42.8below gatenativenativenative71 %91 %
SPX-CD Pro64.6nativenativenative84 %95 %
Messier One v0.242.4below gatenativenativenative85 %85 %
spark-s1-4b-v643.2below gatenativenativenative84 %83 %
KnowLine 31B-Gen177.4nativenativenative91 %97 %
swanOne56.2nativenativenative81 %93 %
AJev lm6q71.5nativenativenative87 %98 %
Quyet-1.0-Medium41.7below gatenativenativenative76 %84 %
Bespoke Nimble-v441.4below gatenativenativenative57 %96 %
Turion Jev 31B70.9nativenativenative87 %95 %
Vansa-3.440.9below gate—————
JPT-4B40.8below gatenativenativenative66 %90 %
lev41.1below gatenativenativenative84 %83 %
NInfer Qwen3.8-Flash-Next mixed49.0below gatenativenativenative75 %93 %
decider-4b v240.1below gatenativenativenative64 %92 %
baselabs mammoth-176.2nativenativenative89 %97 %
Jeffan-4B-201040.1below gatenativenativenative63 %91 %
jev-local41.9below gatenativenativenative85 %82 %
GPT-6 Luna95.7—————
Autoloops – Gemma 4 31B IT58.1—————
Wald 4B41.0below gatenativenativenativenot supported—
GPT-6 Luna96.9—————
ClassOne Qwen 3.5 9B42.2below gatenativenativenative97 %79 %
JevK5 v0.338.5below gatenativenativenative66 %95 %
metask-jev-4b39.2below gatenativenativenative75 %88 %
deck-4B v1.038.6below gatenativenativenative68 %93 %
Kahn1 4B39.7below gatenativenativenative70 %87 %
Clef-Flash42.2below gatenativenativenative68 %93 %
Decision-4B37.9below gatenativenativenative72 %88 %
decisor-4b39.6below gatenativenativenativenot supported—
janus 4B37.6below gatenativenativenative63 %94 %
Qwen3.5-9B Jev-like data-mix v241.9below gatenativenativenative71 %91 %
Seb-9B38.3below gatenativenativenative57 %94 %
JevK5 v0.2.036.3below gatenativenativenative62 %93 %
Tern-35B-A3B44.1below gatenativenativenative63 %96 %
Amos 1 medium38.1below gatenativenativenative56 %97 %
Aleria Judge 1.1.168.0nativenativenative94 %93 %
JevOne46.0below gatenativenativenative62 %96 %
Fastino GLiDE66.1—————
Jebadiah 9B v2.137.2below gatenativenativenative60 %93 %
Verdin-4B35.4below gatenativenativenative65 %86 %
TypeCastLM 1.4.035.4below gatenativenativenative77 %81 %
blink-4b35.0below gatenativenativenative60 %90 %
jiwo 4b34.8below gatenativenativenative63 %90 %
Imajev-4B34.6below gatenativenativenative52 %96 %
SimpleJev Qwen3.6-35B-A3B56.3—————
Uniphore-4B-Decider34.2below gatenativenativenative72 %82 %
Decision 2.0 Lux 9B35.8below gatenativenativenative53 %94 %
Hopper33.6below gatenativenativenative57 %94 %
Malkuth-4B33.8below gatenativenativenative65 %92 %
jqv33.9below gatenativenativenative66 %94 %
JPT-35B-A3B51.4nativenativenative74 %94 %
decider-35b-a3b48.7below gatenativenativenative74 %91 %
CoCo-Decision-4B32.7below gatenativenativenative56 %96 %
Manchego v2.132.5below gatenativenativenative62 %90 %
Decision 4B v1.232.4below gatenativenativenative59 %94 %
Standard One 3B SH v434.4below gatenativenativenative67 %84 %
Fastino GLiDE 1.276.4nativenativenative86 %98 %
Raw Qwen3 4B Instruct 2507 direct logits34.0below gatenativenativenative92 %73 %
StartLux-Decision v1.1 27B Instant73.2nativenativenative82 %99 %
Standard One 3B SH32.8below gatenativenativenative63 %84 %
RSI-Jev v6.1-VL 27B61.6nativenativenative83 %91 %
GPT-5.6 Luna93.0—————
JEV-27B56.8nativenativenative79 %95 %
torchcast-decision-27b70.4nativenativenative83 %97 %
Bespoke Nimble 9B43.1below gatenativenativenative77 %87 %
Nimble-v4-max71.0nativenativenative84 %97 %
pplx-decider-v1.271.7nativenativenative84 %96 %
Perplexity Decider v1.1 27B75.6nativenativenative91 %95 %
vLLM-SR d3-turbo74.8nativenativenative87 %96 %
vLLM-SR d372.9nativenativenative86 %96 %
reflex 4B30.8below gatenativenativenative58 %92 %
Vera Large67.8nativenativenative82 %95 %
Amos 1 large68.2nativenativenative85 %97 %
JADE47.9below gatenativenativenative94 %91 %
Nagi-ENORMOUS 27B62.4nativenativenative84 %91 %
typecastlm29.8below gatenativenativenative57 %90 %
Jebadiah 27B57.5nativenativenative74 %98 %
iota-decide-4b29.4below gatenativenativenative60 %84 %
Decision 2.0 Vega 27B60.4nativenativenative73 %99 %
Jebadiah 9B v230.6below gatenativenativenative52 %93 %
Standard One 8B SH34.5below gatenativenativenative61 %88 %
Decision 4B v1.129.1below gatenativenativenative57 %94 %
AutoJev-27B55.6nativenativenative70 %99 %
pjev-2b28.8below gatenativenativenative59 %88 %
Eikos-27B64.4nativenativenative81 %98 %
Instinct Dual 4B28.3below gate—————
NInfer Qwen3.8-27B NVFP451.8nativenativenative82 %89 %
Compass 0.2.028.9below gatenativenativenative25 %97 %
JEB-35B-A3B33.2below gatenativenativenative54 %94 %
OpenJev70.2nativenativenative96 %93 %
Open-Jev 9B43.9below gatenativenativenative81 %86 %
Gemini 3.1 Flash-Lite58.6—————
Standard One 8B32.8below gatenativenativenative68 %90 %
Clef59.1nativenativenative82 %94 %
ArseneLupin v1.128.0below gatenativenativenative42 %94 %
system-one-open27.9below gate—————
JEV Qwen3.5-9B Base NVFP429.3below gatenativenativenative53 %89 %
Arbiter-4B27.4below gatenativenativenative65 %84 %
Decider-4B27.1below gatenativenativenative54 %90 %
ClassOne Gemma 4 E4B28.2below gatenativenativenative93 %74 %
SemIf, formerly OpenJev27.1below gatenativenativenative62 %90 %
openjev-sglang38.8below gate—————
DecisionNode Edge 0.9.226.8below gatenativenativenative62 %83 %
RYOTIDE-Qwen927.6below gatenativenativenativenot supported—
ClassOne Qwen 3.5 4B27.3below gatenativenativenative86 %71 %
Jez-large-1.0.025.6below gatenativenativenative34 %98 %
RYOTIDE-Qwen25.2below gatenativenativenative57 %89 %
Chon-1.0-12B27.3below gatenativenativenative100 %90 %
Noma25.1below gatenativenativenative51 %88 %
KnowLine 35B-A3B-Gen1.560.3nativenativenative75 %95 %
Bev / Bonsai 27B43.8below gatenativenativenative70 %91 %
LitJev43.3below gatenativenativenative66 %98 %
local-jev Qwen3.5-4B24.1below gatenativenativenative35 %99 %
system-one29.3below gatenativenativenative92 %73 %
StartLux-Decision v1.1 27B AutoThink66.7nativenativenative77 %97 %
kev 8B29.3below gatenativenativenative77 %81 %
Decision 2B22.9below gatenativenativenative39 %94 %
OpenSourceJev23.2below gatenativenativenative49 %95 %
Imajev-9B36.7below gatenativenativenative54 %93 %
open-alternative-jev22.3below gatenativenativenative60 %87 %
Raw Qwen3 8B direct logits26.5below gatenativenativenative92 %71 %
Liquid AI d1-3B22.1below gatenativenativenative49 %83 %
Krino-2B22.0below gatenativenativenative83 %71 %
Imajev-2B21.2below gatenativenativenative38 %84 %
decider-2b20.7below gatenativenativenative68 %78 %
Nemotron Diffusion 8B20.4below gatenativenativenative61 %77 %
DeepSeek V4.1 Flash84.5—————
kev 4B19.5below gatenativenativenative67 %82 %
Clef-omni34.5below gatenativenativenative65 %86 %
Kev 27B64.2nativenativenative80 %97 %
FrontiersMind Lumma-fev-27B55.5nativenativenative73 %93 %
DecisionNode Edge Mini 0.9.217.6below gatenativenativenative47 %79 %
jiwo 0.8b15.9below gatenativenativenative36 %76 %
Malkuth-2B15.9below gatenativenativenative53 %84 %
Qwen3-Reranker-4B16.3below gatenativenativenative25 %75 %
SimpleJev Qwen3.8-27B57.9nativenativenative78 %96 %
ZeroEntropy zerank-215.6below gatenativenativenative14 %87 %
EXAONE-4.0-1.2B-JEV v0.314.8below gatenativenativenative40 %85 %
Open-Jev 2B21.8below gatenativenativenative67 %75 %
SimpleJev Qwen3.8-27B52.9—————
Vega 4B15.7below gatenativenativenative69 %70 %
Open-Jev 27B v1.152.5nativenativenative67 %94 %
Raw Phi-4 mini direct logits13.7below gatenativenativenative50 %61 %
decision-machine-113.1below gate—————
SimpleJev12.7below gatenativenativenative100 %58 %
Jeetu Decider 2B10.2below gatenativenativenative38 %76 %
JevAct10.1below gate—————
smalljev semantic-v98.7below gatenativenativenative23 %73 %
Gutsy 0.8B v0.38.6below gatenativenativenative29 %74 %
Vega-U8.5below gatenativenativenative33 %76 %
PurpleMIST-Mini-1.08.4below gatenativenativenative9 %88 %
kev 0.6B7.6below gatenativenativenative43 %69 %
jul fast7.5below gatenativenativenative30 %89 %
OpenDecision7.4below gatenativenativenative51 %68 %
ClassOne Gemma 4 E2B7.8below gatenativenativenative83 %59 %
Fastino GLiNER-2.5-Decide7.0below gateconfidenceconfidenceconfidence100 %54 %
Deem 0.8B v16.5below gatenativenativenative77 %59 %
Decision Fast6.1below gatenativenativenative14 %90 %
Quyet-1.0-Small-EN6.1below gatenativenativenative28 %64 %
ClassOne Qwen 3.5 2B5.8below gatenativenativenative87 %62 %
Laya typed-decisions5.5below gatenativenativenative1 %100 %
Vega-U5.3below gatenativenativenative20 %53 %
WaterSheep5.3below gatenativenativenative37 %69 %
openJev Verdict 1.45.0below gatenativenativenative0 %—
Raw Qwen3 0.6B direct logits5.6below gatenativenativenative100 %42 %
lev-350m4.8below gatenativenativenative8 %69 %
Raw Qwen3 1.7B direct logits5.1below gatenativenativenative100 %42 %
kev 0.5B4.6below gatenativenativenative26 %56 %
Jez-small-1.0.04.6below gatenativenativenative11 %85 %
openJev Verdict4.1below gatenativenativenative33 %60 %
Quyet-1.0-Small4.0below gatenativenativenative29 %71 %
watt-flash-0.13.8below gatenativenativenative1 %60 %
Quyet-1.0-Tiny3.7below gatenativenativenative20 %64 %
Phocinae-Largha-150M-v13.6below gatenativenativenative3 %55 %
Liquid AI d1-omni-600M3.3below gatenativenativenative27 %56 %
Vega 0.8B3.1below gatenativenativenative32 %63 %
open-jev-deberta-v3-large2.8below gatenativenativenative15 %58 %
Julia-12.4below gatenativenativenative73 %46 %
verdict-small2.3below gatenativenativenative45 %62 %
Tacet Sonata2.2below gatenativenativenative32 %56 %
Laya1.8below gatenativenativenative19 %61 %
Mixedbread mxbai-rerank-base-v21.1below gatenativenativenative0 %—
Laya multilingual0.3below gatenativenativenative72 %50 %
Certo v10.2below gatenativenativenative1 %25 %
CLM-8B0.1below gatenativenativenative51 %69 %
BAAI bge-reranker-v2-m30.1below gatenativenativenative0 %—
Alibaba GTE Reranker ModernBERT-base0.0below gatenativenativenative0 %—
Mirror0.0below gatenativenativenative92 %60 %
Open Jev JSON Canvas61.2labellabellabel100 %85 %
Qwen3.8 27B33.4below gate—————
Microsoft-Decision-1 (Azure Foundry) — A2 re-run59.2nativenativenativenot supported—
All data (305 systems)

Not yet measured on v1.6 · 0 systems with a dated carried score

v1.5 These rows were measured before the 10 s rule (v1.5 method), not re-scored: their scores are not comparable with the re-scored rows above.

Every ranked system of the live v1.5.7 board that is not measured on the v1.6.0 pool keeps its last published score, marked with the release that first published that measurement and that release's publication day. Carried rows are listed separately and never ranked together with v1.6-measured rows. v1.5 protocol (1,624 decisions: 904 open + 720 sealed); scores are on the v1.5 scale and are not comparable with v1.6-measured rows.

Dates: . The date is the publication day of the release that first published the measurement, not a per-model measurement timestamp.

Carried JevBench v1.5.x results, not ranked with v1.6 measurements
SystemMeasured onCapability (v1.5 scale)v1.5 CompositeCost / 1,000Median latency

* Self-hosted rows: the first value is the p50 we measured on our own hardware over loopback (on-GPU, or on-CPU where marked, no network). It is not directly comparable with API latency, which we time end to end from our measuring host, network included. “est.” is the est. incl. network (2x + 150 ms) that the ranking and the latency cap use for every self-hosted row. Model authors can ask us to time a hosted endpoint instead (on request).

Method · v1.7.2

System types (colours)

  • Jev — reference (TypeSafe, closed) — The closed system JevBench is named after, shown as the reference.
  • Closed API (weights not public) — Available through a hosted API; the weights cannot be downloaded.
  • Open weights · LLM decoder — An autoregressive language model with public weights, including fine-tunes, merges and Jev rebuilds.
  • Open weights · diffusion LM — A language model that generates by iterative denoising instead of token by token.
  • Open weights · encoder / classifier — BERT-style encoders, NLI zero-shot classifiers and GLiNER-type models.
  • Open weights · reranker — A cross-encoder or LLM reranker that scores options against the input.
  • Base model control (no decision fine-tune, raw logits) — An official open checkpoint without any decision fine-tune, read out from raw logits, used as a floor.
  • System (router / cascade / ensemble) — Several models combined at inference time.

Colours show architecture only; they never change a score or rank. Each class is assigned from cited evidence (config.json, model card, provider docs, or our own run record for hosted APIs).

Self-hosted rows use S 1,200 + P 300 + L1 (354 items) + L2 (33 items) + L3 (1,118 items), where answered. API rows re-run on A4/A5 use their 300/300-item sealed subset instead of S. Every row’s tag lists the pools it actually answered. L3 is a sealed language supplement (1,118 items), drawn 2026-10-07. Every language has at least 60 items in P ∪ L1 ∪ L2 ∪ L3. Header counts show the full S + P + L1 + L2 + L3 pool. Headline scores are unchanged; these raw cells are unequated, scored for language/category views only and outside the Composite. L3 items were written natively by Claude Sonnet 5.5, each solved blind and language-checked by GPT-6.1 Sol, with gold kept only when both agree or a second review confirms; no gold comes from Jev or any measured API. L3 is API-facing by design and is excluded from future headline draws. Rows with unfinished runs retain their actual coverage tags. L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of OpenAI rows (OpenAI Decisions, GPT-6 Luna, GPT-5.6 Luna) carry that exposure; headline scores do not use L3. C1 adds English items for thin radar categories (everyday language, safety and the other use case). L4 adds 20 new Hindi items for Fastino GLiNER 2.5 Decide only. They were authored with OpenAI and independently solved and language-reviewed with Anthropic before sealing. Seventeen L4 requests succeeded; three HTTP 400 failures remain in the scored observations and do not count as completed coverage. L4 is outside headline and category scores and the common header counts. L5 adds 60 new items for OpenJev DeBERTa v3 Large only: 20 each in Arabic, Hindi and Greek. They were authored with OpenAI and independently solved and language-reviewed with Anthropic before sealing, then checked against the original 512-token native context before keeper selection. All 60 native requests succeeded. Previous failures remain scored and do not count as completed coverage. L5 is outside headline and category scores and the common header counts.

Method amendments

  • 5 Oct 2026, v1.6.1 (Florian's decision): hosted/API systems now answer the same full item set as self-hosted systems (1,200 sealed + 300 public). Their earlier answers on the API subsets were reused and only the missing sealed items were sent; every item was logged in the exposure ledger before it was sent. API rows are no longer equated. Consequence: every v1.6.0 sealed item has now been seen by at least two providers, so the whole v1.6.0 sealed draw is retired for future releases; v1.6.2 onward draws fresh items from the reserve.
  • 5 Oct 2026, v1.6.1 cost rule (Florian's decision): cost per 1,000 decisions is now measured on one common item set for every system: all items except the 23 very long items that lie outside the Jev reference's accepted input range. Before, a system that processed those items paid for their tokens while a system that refused them did not. Refusals still count against Intelligence exactly as before. Hosted APIs with a per-token tariff are priced from their measured tokens on this common set; carried per-decision prices of self-hosted systems are unchanged. Effect: Sage 1.3.0 USD 0.0247 per 1,000 decisions on the common set (0.0650 on all 1,500 items); Jev 1.13.0 0.0323 (unchanged).

Cost rule, generalised in v1.6.30: On every draw, very long items that a ranked row refuses because they exceed its context length are left out of cost for all rows: cost per 1,000 decisions is computed on the same remaining items for every row. A refused item still counts as a wrong answer for Intelligence. First published as the v1.6.1 cost rule (5 Oct 2026; 23 items on that draw). From v1.6.30 it applies to draw v1.6-regular-20261012: the 15 items the Jev reference refused with max_tokens_exceeded are left out and 1,485 remain. Rows of draw v1.6-regular-20261010-a2 still show the all-items cost they were published with until a following release re-costs them under this rule. The 23 items above are the set of the v1.6.1 draw.

The v1.6.1 amendments above say hosted APIs are no longer equated. Exception since v1.7.7: the 17 API rows re-run on A4 ∪ P (600 items) are equated (+2.57 Intelligence, +3.35 Calibration); see “Full API re-run (A4, v1.7.7)” under Open weights and API offerings. Full-set API rows (Jev, Sage, wity-1, Fastino GLiNER-2.5-Decide, d1 (Liquid AI)) are not equated. OpenAI Decisions (gpt-6-luna) ran later on the fresh sealed API set A5 ∪ P (600 items; A4 had already been sent to that provider) and is equated the same way with A5's own offsets (+5.78 Intelligence, +6.15 Calibration, same pool).

Rotating item sets

Each release draws fresh sealed decisions from a larger reserve. Self-hosted open-weights models (run offline on our own GPU pods or Sandy) answer S and P; externally hosted models answer only the API subset A and P.

SetItemsChoiceNoulScoreAnswered by
S · Sealed release draw1,200600300300self-hosted systems (run offline on our own GPU pods or Sandy) and, from v1.6.1, externally hosted APIs
A · API subset (part of S; used by the v1.6.0 hosted-API measurements)3001507575every system (inside S since v1.6.1); retired for future draws
P · Public set3001507575every system

Not in the table above (frozen release data): API set A4 · 300 never-used sealed items, answered together with P by the 18 API endpoints re-run on 6 Oct 2026 (the 17 ranked offerings and the classifier.dev wrapper); retired after that run.

  • Self-hosted systems: 1,500 items (S 1,200 + P 300). Hosted APIs: 1500 items (A 300 + P 300). Exception since v1.7.7: the 17 API rows re-run on A4 ∪ P (600 items) are equated (+2.57 Intelligence, +3.35 Calibration); see “Full API re-run (A4, v1.7.7)” under Open weights and API offerings. Full-set API rows (Jev, Sage, wity-1, Fastino GLiNER-2.5-Decide, d1 (Liquid AI)) are not equated. OpenAI Decisions (gpt-6-luna) ran later on the fresh sealed API set A5 ∪ P (600 items; A4 had already been sent to that provider) and is equated the same way with A5's own offsets (+5.78 Intelligence, +6.15 Calibration, same pool).
  • A sealed item is scored in at most three releases, then retired. An item used in one release is not drawn again in the next. If coverage minimums cannot be met, the release waits for newly reviewed items.
  • The selection seed is committed (SHA-256) before any inference, and the draw is a deterministic function of policy, seed and item id.

API-exposure rule

  • An external exposure is any sealed item sent to an endpoint we do not control: closed APIs, and open-weights models reached through third-party hosts, routers or a submitter's endpoint. Timeouts count as exposure.
  • Hosted models receive only the release&apos;s API subset A plus the public set P, and are re-measured at most once every three refresh releases unless a verified new model version ships. Between measurements they keep their last score with its measurement date.
  • Every externally sent sealed item is logged before dispatch. An item exposed to a provider is never scored again for that provider; once two different providers have received it, it retires for everyone. In v1.6.0 Jev 1.13.0 and Fastino GLiNER-2.5-Decide both received the same A, so A retires globally.

Public-versus-sealed gap penalty

Intelligence is half public, half sealed. A system whose public score exceeds its sealed score by more than the field-median gap (G_med = 2.6 points) plus 8 points loses one Intelligence point per excess point. Hosted APIs are compared on P versus A against the same self-hosted systems' P-versus-A gap (-1.2 points).

Comparable scores for hosted APIs

Median over complete full-coverage self-hosted systems of metric(S u P) - metric(A u P), per metric (Intelligence, Calibration); API value = clip(raw_A + offset). Offsets in this run: Intelligence -2.21, Calibration +2.34. Historical v1.6.0 equating offsets, kept for the record. Hosted APIs that answered the full set (v1.6.1) are not equated. Category and language values stay raw and unequated.

Headline and Composite

Capability = mean(Intelligence, Calibration) for systems within twice the Jev 1.13.0 cost and median latency (the official caps; the sliders change only your view). The Composite (option A) is the equal-weight harmonic mean of Intelligence, Calibration, Speed and Cost with the v1.5 low-axis gates; it remains secondary. Request types Choice, Noul and Score weigh equally; tiers weigh easy 0.1, standard 0.2, hard 0.4, judge 0.3.

Latency. API models are timed end to end from our measuring host, network included. Self-hosted models are timed on our own hardware over loopback (on-GPU, no network), so their measured value is not directly comparable with API latency. The ranking and the latency cap use the same estimate for every self-hosted row: 2 × measured p50 + 150 ms (est. incl. network (2x + 150 ms)). Tables show the measured value first, tagged “on-GPU, no network”, and the estimate second. A model author can provide a hosted endpoint; we then time it exactly like an API, network included, with no assumption (on request: info@productivity-boost.com, or note it in your submission). From JevBench v1.7.0 (method v1.7), a decision that takes longer than 10 seconds (measured time, not the estimate) counts as a wrong answer and as zero calibration for that item; tables show the measured p95 and how many decisions were over 10 s.

Jev-class caps use a fixed reference: Jev 1.13.0 as measured in v1.5 (p50 0.62 s, USD 0.0323 per 1,000 decisions); caps = 2x (1.23 s, USD 0.0646). Jev's own v1.6 p50 is 0.24 s.

Costs of v1.6-measured systems carry each system's published v1.5.4 cost per 1,000 decisions (pricing rules unchanged; v1.6 item lengths differ) unless the row says otherwise; Fastino's is an estimate from its published tariff and measured tokens.

Noul decisiveness and Score baseline (addendum B)

Scored with method option B (scorer setting O1S), selected on 3 Oct 2026 after the v1.6 scores were known and disclosed as a post-results change. Each split × type competence is clipped at 0 before the type weighting, so a type answered no better than chance counts as chance instead of negative. The Score chance baseline is the error of always predicting the mid-scale level, so a flat know-nothing distribution earns about 0. A Score cell whose golds all sit at mid-scale keeps the v1.5 random-level baseline; this only occurs in small breakdown and bootstrap cells. Calibration is unchanged. This run: G_med = 2.57 points; hosted-API offsets Intelligence -2.21, Calibration +2.34.

Failed requests and very long items

23 items of this draw are very long (about 77,000 to 81,000 input tokens). Systems with a shorter context window refuse them; Jev-Omni runs out of GPU memory on them on its listed RTX 6000 (48 GB) recipe; decider-4b v2's server truncates them to about 32,800 tokens and answers; Plumb-4B reads them in full. Each outcome is the system's own recipe and is scored as such. A failed, refused or unparseable answer counts as wrong for Intelligence and stays in the denominator; it does not enter Calibration (the v1.5 rule, applied to every system). Failed answers per system: AJev lm6q 15/1500 · Aleria Judge 1.1.1 15/1500 · Amazon Strands Decider 26B-A4B 15/1500 · Amos 1 large 21/1500 · Amos 1 medium 10/1500 · Aplomb 1 5/1500 · APUS-OpenJev-v1-35B-A3B 23/1500 · APUS-OpenJev-v1-9B 23/1500 · Arcbright Elf 1 12B 15/1500 · Arcbright Elf 1 31B 15/1500 · ArseneLupin v1.1 15/1500 · attest-31B-v1.0 15/1500 · AutoJev-27B 23/1500 · basal-omni-1.0-ultra 15/1500 · BB-Qwen3.5-4B-LoRA 0/1500 · Bespoke Nimble 9B v3 23/1500 · Bev / Bonsai 27B 23/1500 · BAAI bge-reranker-v2-m3 0/1500 · blink-4b 2/1500 · Blink v0.3 26B-A4B 29/1500 · Bobcat 1.3 15/1500 · Bobcat Flash 1.2 13/1500 · Bongard-mini 15/1500 · Celeris-1 Decision 0/1500 · Certo v1 0/1500 · Chon-1.0-12B 540/1500 · Chon-1.0-12B-Fast 0/1500 · Chon-1.0-E4B 0/1500 · classifier.dev 9/600 · ClassOne Gemma 4 E2B 23/1500 · ClassOne Gemma 4 E4B 29/1500 · ClassOne Qwen 3.5 2B 29/1500 · ClassOne Qwen 3.5 4B 29/1500 · ClassOne Qwen 3.5 9B 23/1500 · Clef 0/1500 · Clef-Flash 0/1500 · Clef-omni 0/1500 · CLM-8B 1/1500 · CoCo-Decision-4B 23/1500 · Compass 0.2.0 0/1500 · Cygnet 23/1500 · decider-12b v2 23/1500 · decider-12b v1, stock Gemma-4-12B-it 23/1500 · decider-2b 0/1500 · decider-35b-a3b 0/1500 · decider-4b v2 0/1500 · decider chat on Gemma-4-31B-it 23/1500 · decisio v0.8.0 on gemma-4-12B-it 23/1500 · decisio v0.9.0 on gemma-4-12B-it 23/1500 · decisio v0.8.0 on gemma-4-31B-it 23/1500 · Decision 2.0 Vega 27B 23/1500 · Decision 2B 23/1500 · Decision 4B v1.1 23/1500 · Decision 4B v1.2 23/1500 · Decision Fast 23/1500 · decision-machine-1 9/600 · Decision 2.0 Lux 9B 15/1500 · DecisionNode Edge 0.9.2 15/1500 · DecisionNode Edge Mini 0.9.2 15/1500 · decisor-4b 0/1500 · deck-4B v1.0 23/1500 · deck-31B 23/1500 · Deem 0.8B v1 23/1500 · DeepSeek V4.1 Flash 5/600 · Diffusion Jev 26/1500 · Drex Max 2/1500 · Eikos-27B 23/1500 · baselabs elephant-1 15/1500 · Decision-4B 23/1500 · EXAONE-4.0-1.2B-JEV v0.3 23/1500 · Fastino GLiDE 9/600 · Fastino GLiDE 1.2 15/1497 · Fastino GLiDE (hosted API, 22 Oct 2026) 19/1497 · Fastino GLiNER-2.5-Decide 31/1500 · Gemini 3.1 Flash-Lite 1/600 · GEV-26B-Decide 23/1500 · Arbiter-4B 15/1500 · GPT-5.6 Luna 0/600 · GPT-6 Luna (default medium reasoning effort) 4/600 · GPT-6 Luna (low reasoning effort) 0/600 · Alibaba GTE Reranker ModernBERT-base 0/1500 · Gutsy 0.8B v0.3 29/1500 · H2O-Lightning-31B v1.0.7 15/1500 · H2O-Lightning-4B v1.1 0/1500 · Hopper 22/1500 · Hopper 12B trained 23/1500 · HS1 0.1.2 15/1500 · Imajev-2B 15/1500 · Imajev-4B 23/1500 · Imajev-9B 15/1500 · Instinct 9/600 · Instinct Dual 4B 0/600 · iota-decide-4b 0/1500 · JADE 392/1500 · janus 4B 23/1500 · JEB-35B-A3B 15/1500 · Jebadiah 27B 24/1500 · Jebadiah 9B v2 15/1500 · Jebadiah 9B v2.1 15/1500 · Jeetu Decider 2B 15/1500 · Jeff-1.0-Large 21/1500 · Jeffan-31b-1710 15/1500 · Jeffan-4B-2010 15/1500 · Jev 1.13.0 23/1500 · Jev 1.14.0 15/1500 · JEV-27B 23/1500 · jev-local 23/1500 · Jev-Omni 23/1500 · JEV Qwen3.5-9B Base NVFP4 23/1500 · JevAct 21/600 · JevK5 v0.3 23/1500 · JevK5 v0.2.0 23/1500 · JevOne 0/1500 · jevora-9b-v1 49/1500 · Jez-large-1.0.0 15/1500 · Jez-small-1.0.0 15/1500 · jiwo 0.8b 29/1500 · jiwo 4b 29/1500 · JPT-35B-A3B 23/1500 · JPT-4B 15/1500 · jqv 23/1500 · jul fast 6/1500 · Julia-1 17/1500 · Kahn1 4B 23/1500 · kettei-chub 15/1500 · kettei-fat 15/1500 · kev 0.5B 32/1500 · kev 0.6B 30/1500 · Kev 27B 23/1500 · kev 4B 29/1500 · kev 8B 25/1500 · Kev-9B v2 15/1500 · KnowLine 31B-Gen1 15/1500 · KnowLine 35B-A3B-Gen1.5 0/1500 · Krino-2B 0/1500 · Autoloops – Gemma 4 31B IT 9/600 · Myles-G 12B 0/1500 · Myles-G 31B 0/1500 · Laya 0/1500 · Laya multilingual 0/1500 · Laya typed-decisions 0/1500 · lev 25/1500 · lev-350m 23/1500 · d1 0/1500 · Liquid AI d1-3B 0/1500 · Liquid AI d1-omni-600M 0/1500 · d1-ultra 15/1500 · LitJev 23/1500 · local-jev Qwen3.5-4B 0/1500 · FrontiersMind Lumma-fev-27B 15/1500 · Malkuth-2B 23/1500 · Malkuth-4B 23/1500 · baselabs mammoth-1 15/1500 · Manchego v2.1 23/1500 · Mercury Decide 23/1500 · Messier One v0.2 0/1500 · Messier One v0.4 0/1500 · metask-jev-4b 23/1500 · Metask rain 4B 0/1500 · metask-jev-rain-12B 0/1500 · metask-jev-rain-12B-trained 12/1500 · metask-jev-rain-31B-trained 0/1500 · metask-jev-rain-31B-trained service v3 0/1500 · Qwen3.5-0.8B Decision Model 0/1500 · Microsoft-Decision-1 0/0 · Microsoft-Decision-1 (Azure Foundry) — A2 re-run 7/600 · Mirror 592/1500 · modernbert-jev v2 0/1500 · Mixedbread mxbai-rerank-base-v2 0/1500 · Nagi-ENORMOUS 27B 15/1500 · Nagi-HUGE 12B 15/1500 · Nemotron Deciffusion 8B v2 0/1500 · Nemotron Diffusion 8B 23/1500 · Nev-2B 15/1500 · Bespoke Nimble 9B 23/1500 · Bespoke Nimble-v4 0/1500 · Bespoke Nimble-v4-max 0/1500 · Nimble-v4-max 16/1500 · NInfer Qwen3.8-27B NVFP4 23/1500 · NInfer Qwen3.8-Flash-Next mixed 23/1500 · Noma 0/1500 · open-alternative-jev 0/1500 · open-jev-deberta-v3-large 73/1500 · Open Jev JSON Canvas 23/1500 · Open-Jev 27B v1.1 23/1500 · Open-Jev 2B 23/1500 · Open-Jev 9B 23/1500 · OpenAI Decisions 2/600 · OpenDecision 0/1500 · openjev-sglang 9/600 · OpenJev 24/1500 · openJev Verdict 0/1500 · openJev Verdict 1.4 0/1500 · OpenSourceJev 23/1500 · Phocinae-Largha-150M-v1 375/1500 · pjev-2b 15/1500 · Plumb-4B 2/1500 · pplx-decider-flash-v1.2 15/1500 · Perplexity Decider v1.1 27B 24/1500 · pplx-decider-v1.2 15/1500 · PurpleMIST-Flash-1.0 15/1500 · PurpleMIST-Mini-1.0 15/1500 · Quyet-1.0-Large 0/1500 · Quyet-1.0-Medium 0/1500 · Quyet-1.0-Small 0/1500 · Quyet-1.0-Small-EN 0/1500 · Quyet-1.0-Tiny 0/1500 · Qwen3-Reranker-4B 0/1500 · Qwen3.8 27B 8/600 · Qwen3.5-9B Jev-like data-mix v2 23/1500 · Raw Phi-4 mini direct logits 23/1500 · Raw Qwen3 0.6B direct logits 23/1500 · Raw Qwen3 1.7B direct logits 23/1500 · Raw Qwen3 4B Instruct 2507 direct logits 23/1500 · Raw Qwen3 8B direct logits 23/1500 · reflex 4B 2/1500 · Reflex Instinct 0.6B 0/1500 · René-1 31B FP8 23/1500 · Respan Span-02 15/1500 · Decider-4B 15/1500 · RSI-Jev v6.1-VL 27B 29/1500 · RYOTIDE-Gemma 15/1500 · RYOTIDE-Qwen 2/1500 · RYOTIDE-Qwen9 21/1500 · Sage 1.3.0 0/1500 · Sage 1.4 30/1500 · Seb-9B 15/1500 · Seb-9B-v2 15/1500 · SemIf, formerly OpenJev 23/1500 · shisa-de-1 16/1500 · SimpleJev 37/1500 · SimpleJev Qwen3.6-35B-A3B 9/600 · SimpleJev Qwen3.8-27B 9/600 · SimpleJev Qwen3.8-27B (self-hosted, v1.6 pool) 23/1500 · Super Decider 11/1500 · Super Decider (thinking) 4/1500 · smalljev semantic-v9 0/1500 · spark-s1-4b-v6 0/1500 · SPX-CD Flash 23/1500 · SPX-CD-Omni 23/1500 · SPX-CD Pro 24/1500 · Standard One 3B SH 15/1500 · Standard One 3B SH v4 15/1500 · Standard One 8B SH 29/1500 · Standard One 8B 23/1500 · StartLux-Decision v1.1 27B AutoThink 27/1500 · StartLux-Decision v1.1 27B Instant 15/1500 · SubQ Snap 1.1 0/1500 · SubQ Snap 0/1500 · Surogate Rune 26B-A4B v3 23/1500 · swanOne 23/1500 · system-one-open 0/600 · system-one 23/1500 · Tacet Sonata 3/1500 · TDE general large v0.1 0/1500 · Tern-35B-A3B 15/1500 · torchcast-decision-12b 23/1500 · torchcast-decision-27b 23/1500 · Torchcast Decision 4B (v1.1) 0/1500 · Torchcast Decision 4B (v1.2) 0/1500 · Torchcast Decision 4B (v1.3) 0/1500 · Trio-Spark v1.3 102/1500 · Turion Jev 31B 15/1500 · typecastlm 0/1500 · TypeCastLM 1.4.0 0/1500 · Uniphore-4B-Decider 15/1500 · Vansa-3.4 9/600 · Vega-U (Qwen3.5-0.8B reader) 15/1500 · Vega-U (Qwen3.5-4B reader) 15/1500 · Vega 0.8B 0/1500 · Vega 4B 29/1500 · Vera Large 15/1500 · verdict-small 0/1500 · Verdin-4B 15/1500 · vLLM-SR d3 15/1500 · vLLM-SR d3-mini 15/1500 · vLLM-SR d3-turbo 15/1500 · Wald 26B-A4B 0/1500 · Wald 4B 8/1500 · Wald 4B v2.1 0/1500 · WaterSheep 1/1500 · watt-flash-0.1 23/1500 · Winnow-12B Q8 23/1500 · wity-1 (Wity, reasoning auto) 36/1500 · wity-1 (Wity, reasoning always) 36/1500 · wity-1 (Wity, reasoning off) 0/1500 · Xevn-4B 15/1500 · Xor 26B-A4B 23/1500 · ZeroEntropy zerank-2 0/1500.

Calibration basis

Systems that return a full probability distribution are calibrated on all components (top-label error, plus distribution distance for Choice and ranked-probability error for Score). Fastino GLiNER-2.5-Decide returns a single confidence value, so its Calibration is the top-label error only and is not like-for-like with full-distribution systems.

Overnight full re-measure (4–5 Oct 2026)

Every system with a reproducible recipe was re-run on the v1.6.0 pool overnight with the same pinned inputs and scorer (method option B / O1S). This page uses scoring round score-v161-3 (v1.6.1) (2026-10-06 00:39:36 UTC). Only complete runs (1,500 items self-hosted, the full API input for hosted APIs) are ranked; partial runs are never ranked, and systems not yet re-measured keep their dated v1.5.x score in the separate table.

  • Scores are the official v1.6.0 scorer (score_v16.py, method option B / O1S, bootstrap B = 1,000) over complete outputs only; each scoring round is kept separately.
  • Hosted APIs are scored on the same full item set as self-hosted systems (S 1,200 + P 300, v1.6.1) and are not equated.
  • GPU-class deviations: reproducible recipes used the hardware listed per row, including H100 for large fast-lane decoders and RTX 6000 for the baseline; hardware differences remain in measured latency. The standard x2 + 0.15 s self-hosted adjustment is an assumption, not a hardware normalization.
  • Wity auto remains outside the latency cap and stays ranked in Composite A. OFF and ALWAYS are unranked variants of the AUTO main row.
  • 6 further measured candidates await a separate publication decision.
  • Sage 1.3.0 (Levanto Labs) was measured on 5 Oct from Sandy (Helsinki), text only. Cost uses the Levanto list tariff (USD 0.05/M input, USD 10/M output; levanto.ai/pricing, read 5 Oct) over all 600 answered rows: USD 0.0766/1,000 decisions. Superseded by the v1.6.1 cost rule (common item set): USD 0.0247/1,000 decisions, inside the Jev-class cost cap.
  • A fresh Monday Jev 1.13.0 re-check on A3 measured p50 0.239 s and equated Capability 77.5 versus the official 76.5, within the confidence interval; the official v1.6.0 Jev row is retained.

Display note on the overnight text above (frozen release data). Exception since v1.7.7: the 17 API rows re-run on A4 ∪ P (600 items) are equated (+2.57 Intelligence, +3.35 Calibration); see “Full API re-run (A4, v1.7.7)” under Open weights and API offerings. Full-set API rows (Jev, Sage, wity-1, Fastino GLiNER-2.5-Decide, d1 (Liquid AI)) are not equated. OpenAI Decisions (gpt-6-luna) ran later on the fresh sealed API set A5 ∪ P (600 items; A4 had already been sent to that provider) and is equated the same way with A5's own offsets (+5.78 Intelligence, +6.15 Calibration, same pool).

Supplementary API draws A2 and A3

History: hosted APIs were first measured on API subsets (A, then A2 and A3) and equated to the self-hosted scale in v1.6.0. From v1.6.1 they answer the same full set as self-hosted systems, so no equating is applied; the earlier subsets A, A2 and A3 are retired.

Per-model exposure counts (hosted and author-hosted endpoints)

System (provider)v1.6 sealed items sentScored sealed setStatus
classifier-dev-fast (classifier.dev)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
decision-machine-1 (milliseconds.ai)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
deepseek-flash (deepseek)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
fastino-gliner-2-5-decide (fastino)1200S (1,200 sealed) + P (300 public)full set since v1.6.1 (5 Oct 2026); earlier API-subset answers reused, only missing sealed items sent; whole v1.6.0 sealed draw retired
gemini-3.1-flash-lite (google)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
gpt-5.6-luna (openai)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
gpt-6-luna (openai)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
gpt-6-luna-low (openai)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
instinct (zoowork)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
instinct-dual-4b (zoowork)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
jev-1.13.0 (typesafe)1200S (1,200 sealed) + P (300 public)full set since v1.6.1 (5 Oct 2026); earlier API-subset answers reused, only missing sealed items sent; whole v1.6.0 sealed draw retired
jevact (jevact)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
kushal-gemma4-31b-it-autoloops (autoloops)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
openjev-sglang (modal via modal)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
qwen3.8-27b (chutes)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
Sage 1.3.0 (Levanto Labs)1200S (1,200 sealed) + P (300 public)full set since v1.6.1 (5 Oct 2026); earlier API-subset answers reused, only missing sealed items sent; whole v1.6.0 sealed draw retired
simplejev-qwen3.6-35b-a3b (featherless)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
simplejev-qwen3.8-27b (featherless)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
system-one-open (modal via modal)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
vansa-3.4 (vansa)300A4 (retired after this run)measured 6 Oct 2026 on A4 ∪ P (v1.7.7)
wity-1-always (wity via railway)1200S (1,200 sealed) + P (300 public)full set since v1.6.1 (5 Oct 2026); earlier API-subset answers reused, only missing sealed items sent; whole v1.6.0 sealed draw retired
wity-1-auto (wity via railway)1200S (1,200 sealed) + P (300 public)full set since v1.6.1 (5 Oct 2026); earlier API-subset answers reused, only missing sealed items sent; whole v1.6.0 sealed draw retired
wity-1-off (wity via railway)1200S (1,200 sealed) + P (300 public)full set since v1.6.1 (5 Oct 2026); earlier API-subset answers reused, only missing sealed items sent; whole v1.6.0 sealed draw retired

Revision history

  • v1.6.1 · 2026-10-05 · Hosted APIs (fastino-gliner-2-5-decide, jev-1.13.0, sage-1.3.0, wity-1, wity-1-always, wity-1-off) answer the full 1,500-item set instead of the 600-item API subset and are no longer equated; self-hosted rows unchanged; whole v1.6.0 sealed draw retired. Cost per 1,000 decisions is measured on one common item set (all items except the 23 outside the Jev reference input range) for every token-priced row.
  • v1.6.0 · 2026-10-05 · Rotating item draw (1,200 sealed + 300 public); hosted APIs on API subsets equated to the self-hosted scale; /jev-models/v1.6.0 stays available unchanged.

Provenance

Measured on the v1.6 pool (run completion day, UTC): 2026-10-02: AutoJev-27B, BAAI bge-reranker-v2-m3, Certo v1, CLM-8B, Cygnet, decider-2b, decider-35b-a3b, decider-4b v2, Decision 4B v1.1, Decision 4B v1.2, Eikos-27B, Alibaba GTE Reranker ModernBERT-base, Hopper, Imajev-4B, jev-local, Jev-Omni, JEV Qwen3.5-9B Base NVFP4, JevK5 v0.3, JevK5 v0.2.0, JevOne, jqv, kev 0.5B, kev 0.6B, kev 4B, kev 8B, Laya, LitJev, local-jev Qwen3.5-4B, Malkuth-2B, Malkuth-4B, metask-jev-4b, Qwen3.5-0.8B Decision Model, Mirror, Mixedbread mxbai-rerank-base-v2, Bespoke Nimble 9B, NInfer Qwen3.8-27B NVFP4, NInfer Qwen3.8-Flash-Next mixed, open-alternative-jev, open-jev-deberta-v3-large, Open Jev JSON Canvas, Open-Jev 2B, Open-Jev 9B, OpenDecision, OpenJev, openJev Verdict, openJev Verdict 1.4, OpenSourceJev, Plumb-4B, Qwen3-Reranker-4B, Qwen3.5-9B Jev-like data-mix v2, Raw Phi-4 mini direct logits, Raw Qwen3 0.6B direct logits, Raw Qwen3 1.7B direct logits, Raw Qwen3 4B Instruct 2507 direct logits, Raw Qwen3 8B direct logits, reflex 4B, SemIf, formerly OpenJev, SimpleJev, smalljev semantic-v9, Standard One 8B, swanOne, system-one, typecastlm, verdict-small, Winnow-12B Q8, ZeroEntropy zerank-2 · 2026-10-04: Bev / Bonsai 27B, Clef, Clef-Flash, decider-12b v2, decider-12b v1, stock Gemma-4-12B-it, Deem 0.8B v1, Decision-4B, H2O-Lightning-4B v1.1, janus 4B, Laya multilingual, Laya typed-decisions, lev, Manchego v2.1, Nemotron Diffusion 8B, Open-Jev 27B v1.1, Quyet-1.0-Large, Quyet-1.0-Medium, Quyet-1.0-Small, Quyet-1.0-Small-EN, Quyet-1.0-Tiny, torchcast-decision-12b · 2026-10-05: Decision 2B, Decision Fast, deck-4B v1.0, deck-31B, Fastino GLiNER-2.5-Decide, Jev 1.13.0, lev-350m, Sage 1.3.0, spark-s1-4b-v6 · 2026-10-06: Aplomb 1, Bespoke Nimble 9B v3, classifier.dev, decider chat on Gemma-4-31B-it, decisio v0.8.0 on gemma-4-12B-it, decisio v0.8.0 on gemma-4-31B-it, Decision 2.0 Vega 27B, decision-machine-1, DeepSeek V4.1 Flash, Diffusion Jev, Fastino GLiDE, Gemini 3.1 Flash-Lite, GEV-26B-Decide, GPT-5.6 Luna, GPT-6 Luna (default medium reasoning effort), GPT-6 Luna (low reasoning effort), Hopper 12B trained, Instinct, Instinct Dual 4B, JADE, Jebadiah 27B, JEV-27B, JevAct, JPT-35B-A3B, Kev 27B, Autoloops – Gemma 4 31B IT, d1, OpenAI Decisions, openjev-sglang, Perplexity Decider v1.1 27B, Qwen3.8 27B, SimpleJev Qwen3.6-35B-A3B, SimpleJev Qwen3.8-27B, SimpleJev Qwen3.8-27B (self-hosted, v1.6 pool), SPX-CD Flash, SPX-CD-Omni, SPX-CD Pro, Surogate Rune 26B-A4B v3, system-one-open, torchcast-decision-27b, Vansa-3.4, wity-1 (Wity, reasoning auto), wity-1 (Wity, reasoning always), wity-1 (Wity, reasoning off), Xor 26B-A4B · 2026-10-07: APUS-OpenJev-v1-35B-A3B, APUS-OpenJev-v1-9B, BB-Qwen3.5-4B-LoRA, Bobcat Flash 1.2, ClassOne Gemma 4 E2B, ClassOne Qwen 3.5 9B, CoCo-Decision-4B, decisio v0.9.0 on gemma-4-12B-it, EXAONE-4.0-1.2B-JEV v0.3, jul fast, Kahn1 4B, Mercury Decide, Messier One v0.2, René-1 31B FP8, Tacet Sonata, TypeCastLM 1.4.0, WaterSheep, watt-flash-0.1 · 2026-10-09: decisor-4b, Jeff-1.0-Large, Metask rain 4B, metask-jev-rain-12B, Wald 4B · 2026-10-10: Blink v0.3 26B-A4B, ClassOne Gemma 4 E4B, ClassOne Qwen 3.5 2B, ClassOne Qwen 3.5 4B, Clef-omni, Gutsy 0.8B v0.3, jiwo 0.8b, jiwo 4b, Liquid AI d1-3B, Liquid AI d1-omni-600M, Microsoft-Decision-1, modernbert-jev v2, RSI-Jev v6.1-VL 27B, RYOTIDE-Qwen9, Standard One 8B SH, Vega 0.8B, Vega 4B, Wald 4B v2.1 · 2026-10-12: attest-31B-v1.0, Krino-2B, metask-jev-rain-12B-trained, metask-jev-rain-31B-trained, Microsoft-Decision-1 (Azure Foundry) — A2 re-run, Standard One 3B SH, Torchcast Decision 4B (v1.1), Uniphore-4B-Decider, vLLM-SR d3 · 2026-10-13: Amos 1 large, Amos 1 medium, Julia-1, metask-jev-rain-31B-trained service v3, Phocinae-Largha-150M-v1, Reflex Instinct 0.6B, Sage 1.4, SubQ Snap · 2026-10-14: Messier One v0.4, Bespoke Nimble-v4, Bespoke Nimble-v4-max, Respan Span-02 · 2026-10-15: d1-ultra, Nemotron Deciffusion 8B v2, Nev-2B, Trio-Spark v1.3 · 2026-10-16: ArseneLupin v1.1, blink-4b, Bongard-mini, Celeris-1 Decision, Compass 0.2.0, Decision 2.0 Lux 9B, Imajev-2B, Imajev-9B, JEB-35B-A3B, Jebadiah 9B v2, Jebadiah 9B v2.1, Jev 1.14.0, jevora-9b-v1, JPT-4B, kettei-chub, kettei-fat, Kev-9B v2, Nagi-ENORMOUS 27B, Nagi-HUGE 12B, Noma, RYOTIDE-Gemma, RYOTIDE-Qwen, shisa-de-1, StartLux-Decision v1.1 27B AutoThink, StartLux-Decision v1.1 27B Instant, TDE general large v0.1 · 2026-10-17: basal-omni-1.0-ultra, PurpleMIST-Flash-1.0 · 2026-10-18: Torchcast Decision 4B (v1.2), Vega-U (Qwen3.5-0.8B reader) · 2026-10-19: Chon-1.0-12B, Chon-1.0-E4B, pplx-decider-flash-v1.2, vLLM-SR d3-turbo · 2026-10-20: Arcbright Elf 1 12B, pplx-decider-v1.2, Seb-9B · 2026-10-21: Drex Max, Arbiter-4B, HS1 0.1.2, iota-decide-4b, Myles-G 12B, Myles-G 31B, Nimble-v4-max, Decider-4B, Super Decider, Super Decider (thinking) · 2026-10-22: AJev lm6q, Aleria Judge 1.1.1, Arcbright Elf 1 31B, baselabs elephant-1, Fastino GLiDE (hosted API, 22 Oct 2026), H2O-Lightning-31B v1.0.7, Jez-large-1.0.0, Jez-small-1.0.0, FrontiersMind Lumma-fev-27B, baselabs mammoth-1, pjev-2b, PurpleMIST-Mini-1.0, SubQ Snap 1.1, Tern-35B-A3B, Verdin-4B · 2026-10-23: Bobcat 1.3, Fastino GLiDE 1.2, KnowLine 35B-A3B-Gen1.5, Seb-9B-v2, Standard One 3B SH v4, Torchcast Decision 4B (v1.3), Vega-U (Qwen3.5-4B reader), Wald 26B-A4B · 2026-10-24: Amazon Strands Decider 26B-A4B, DecisionNode Edge 0.9.2, DecisionNode Edge Mini 0.9.2, Jeffan-31b-1710, Jeffan-4B-2010, KnowLine 31B-Gen1, Turion Jev 31B, Vera Large, vLLM-SR d3-mini, Xevn-4B · 2026-10-25: Chon-1.0-12B-Fast, Jeetu Decider 2B.

Aggregate files: results sha256 754f776d1a14660936eea464070991243b0049aebda140ea5ff25c590e4fe2a7 · categories sha256 fdb4d9dcc4eb625bfdffb3fe6b113f0239f19a77773987cbad64ca94185ea8eb · dated carry sha256 271d1fc77a136a04a1eb6deb5480c52d4211eb2b66b2ac8bb31e80867e69003a. Scoring source sha256 4ab60edc7f3dd88cfbecd4f4b3564d2ba0e98d82e36ea0079f87e99a5db8e6a2. The method, release data and carry artifact are independently hashable.

What changed in v1.7.2

Method v1.7: from v1.7.0 a decision that takes longer than 10 seconds (the measured time, strictly more than 10.000 s) counts as a wrong answer and as zero calibration for that item (each item type's calibration is multiplied by the share of its decisions answered within 10 s). Every row is re-scored from its recorded per-decision times: Intelligence, Calibration, Capability, the composites and their intervals are recomputed with the same frozen scorer plus the rule; the Speed axis now uses all decisions given (it replaces the 124-item subset of v1.6). Cost, tokens, the draws, the raw outputs and the field median are unchanged; no model was run again for the re-scored rows. Every re-scored row shows its v1.6 values next to the new ones.

Time limit on measured time: the limit uses the measured latency, not the displayed estimate (x2 + 0.15 s for self-hosted rows), so a self-hosted row can show an adjusted p95 above 10 s with no decision over the limit. In total 1,455 decisions across 59 rows are over the limit.

Re-measure pending: 5 rows (chon-1.0-e4b, nev-2b, modernbert-jev-v2, mghafiri-qwen3.5-0.8b-decision-model, reflex-instinct-0.6b) recorded their per-decision times on a loaded CPU host, so they cannot be scored under the time limit and no estimate is published. They stay on the board with their v1.6 values, labelled as v1.6, unranked, until they are measured again.

Re-measured in v1.7.0: 4 rows that were pending (loaded CPU host) were measured again on the same items on a dedicated GPU. Each GPU row replaces the earlier CPU row in full, scores and times; GPU times are never combined with CPU scores, and the CPU values stay visible as the row's v1.6 values. pjev-2b (Kailune AI, frozen MiniCPM5-2B + merged LoRA r64, letter-logit readout, T=1 stub) (pjev-2b): no decision took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw. PurpleMIST-Mini-1.0 (Olaverse) (purplemist-mini-1.0): no decision took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw. Vega-U (Qwen3.5-4B reader) (vega-u-qwen3.5-4b): 15 of its 1,500 decisions took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw. Verdin-4B (Victor Bennett, Qwen3.5-4B + merged LoRA r16 + pointer head) (verdin-4b): no decision took longer than 10 seconds; re-measured on the same items, second and last use of these weights on the draw.

Open/sealed gap penalty: the public-versus-sealed gap penalty is computed from the row's own results, so a decision that now fails can slightly reduce that penalty and raise Intelligence a little. This affects 1 row in this release (shisa-de-1 (shisa-ai, Gemma-4-26B-A4B): +0.006 Intelligence points); it is a side effect of the existing penalty, not a change to it.

Dated carry: 14 rows measured before v1.6 (v1.5 method) are carried as published and not re-scored.

Equated API rows: the pool re-runs behind the A4 and A5 equating offsets are scored under the same rule, so the offsets moved with them. These rows keep the 2-decimal precision of their earlier publication.

New in v1.7.0: 26 rows measured for the first time under the 10 s rule. AJev lm6q (Gemma 4 31B) (ajev-gemma4-31b-lm6q): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Aleria Judge 1.1.1 (ALERIA-ML, inference technique on stock Gemma 4 31B, author server, self-hosted) (aleria-judge-1.1.1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Arcbright Elf 1 31B (arcbright-elf-1-31b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Bobcat 1.3 (Gemma 4 31B) (bobcat-1.3): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. DecisionNode Edge 0.9.2 (decisionnode-edge-0.9.2): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. DecisionNode Edge Mini 0.9.2 (decisionnode-edge-mini-0.9.2): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. baselabs elephant-1 (9B) (elephant-1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Fastino GLiDE 1.2 (open weights, self-hosted) (fastino-glide-1.2-open): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Jeffan-31b-1710 (jeffan-31b-1710): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Jeffan-4B-2010 (jeffan-4b-2010): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Jez-large-1.0.0 (pragmaticfrontier, Gemma-4-E4B label-logit decision model, author server, self-hosted) (jez-large-1.0.0): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Jez-small-1.0.0 (pragmaticfrontier, Qwen3.5-0.8B label-logit decision model, author server, self-hosted) (jez-small-1.0.0): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. KnowLine 31B-Gen1 (knowline-31b-gen1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. KnowLine 35B-A3B-Gen1.5 (knowline-35b-a3b-gen1.5): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. FrontiersMind Lumma-fev-27B (lumma-fev-27b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. baselabs mammoth-1 (27B) (mammoth-1): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Nimble-v4-max (open weights, self-hosted) (nimble-v4-max-open): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 1 of its 1,500 decision took longer than 10 seconds and is scored by the rule. Its mandatory disclosure and licence note are shown on the row. Seb-9B-v2 (ironbcc, Qwen3.5-9B fine-tune, one forward pass, stock vLLM, self-hosted) (seb-9b-v2): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Super Decider (singularity-super-decider): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 11 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Super Decider (thinking) (singularity-super-decider-thinking): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 4 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Standard One 3B SH v4 (Standard Thinking) (standard_one_3b_sh_v4): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Tern-35B-A3B (matthewagi, Qwen3.6 35B-A3B MoE decision model, author server, self-hosted) (tern-35b-a3b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Vera Large (vera-large): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. vLLM-SR d3-mini (vllm-sr-d3-mini): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; 15 of its 1,500 decisions took longer than 10 seconds and are scored by the rule. Its mandatory disclosure and licence note are shown on the row. Wald 26B-A4B (Gemma 4 26B-A4B) (wald-26b-a4b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row. Xevn-4B (xevn-4b): measured for the first time under the 10 s rule, so it has no v1.6 values and no old-versus-new comparison; no decision took longer than 10 seconds. Its mandatory disclosure and licence note are shown on the row.

Board effect: the 10 s rule (method v1.7) re-scored every row; the merged-board top fives before/after v1.7.0 were compared over all 36 mixed, open-weights and API variants (including cost caps of 7x and 10x the reference cost, each also with no latency cap). The reference cost is that of Jev 1.13.0, USD 0.0323 per 1,000 decisions, and the official cap is 2x that. Together with 4 release-board lists, 37 of 40 top-five lists change. Top-five changes caused by: the 10 s rule (method v1.7) and the new rows arcbright-elf-1-31b, fastino-glide-1.2-open, knowline-31b-gen1, mammoth-1, singularity-super-decider, singularity-super-decider-thinking (measured for the first time in v1.7.0); entering a top five: arcbright-elf-1-31b, fastino-glide-1.2-open, kettei-fat, knowline-31b-gen1, mammoth-1, singularity-super-decider, singularity-super-decider-thinking; leaving a top five: chon-1.0-12b, drex-max, fastino-glide-api, h2o-lightning-31b, liquid-d1-ultra, nimble-v4-max, qwen3.8-27b, subq-snap-1.1, subq-snap-preview, vllm-sr-d3, vllm-sr-d3-turbo.

Publication of v1.7.0: Florian selected "Publish (recommended)" on Telegram card #18506 (2026-10-24T14:13:05Z), the top-five preview card with preview PNG sha256 b082a945643c70be13f1628cfb9a908f4d11f1acdc72e33b2785ffecad617d2e. Florian top-five preview approval: Telegram card #18506; selected "Publish (recommended)" (2026-10-24T14:13:05Z); preview PNG SHA-256 b082a945643c70be13f1628cfb9a908f4d11f1acdc72e33b2785ffecad617d2e.

Three new rows on v1.6-regular-20261012: Amazon Strands Decider 26B-A4B, Turion JEV 31B and Chon-1.0-12B-Fast. No method change. All 285 carried rows retain every score, confidence interval and diagnostic; only placements move. New rows use their already-scored full-field B=1000 intervals and the 10-second share rule.

One new row on v1.6-regular-20261012: Jeetu Decider 2B. No method change. All 288 carried rows retain every score, confidence interval and diagnostic; only placements move. The new row uses its already-scored full-field B=1000 intervals and the 10-second share rule.

Historical v1.3.0 board: weightings, per-task grid, topic radars and held-out diagnostics

Open this section to load the earlier public-only board and diagnostics.