JevBench v1.7.9: one same-draw self-hosted addition and one price correction

JevBench v1.7.9: one same-draw self-hosted addition and one price correction

One new self-hosted row GLiClass v4 9B; Myles-G 12B now priced at its public list price; Super Decider links point to the vendor page; no method change. GLiClass answered 1,485 of 1,500 requests; 15 HTTP 422 length refusals count as misses. Its cost is an ESTIMATE from Qwen/Qwen3.5-9B with zero generated output. Myles-G 12B uses GBP 0.0015 per million input tokens, output free; prices exclude VAT. Price corrected in v1.7.9; earlier releases showed a same-base ESTIMATE of USD 0.0264 per 1,000. All other scores and confidence intervals are unchanged.

Aggregate results JSON · SHA-256 ab863afda444baa950b6007768d0048060e5661b019ac94b0a777d6dded63294 · Previous v1.7.8

JevBench v1.7.9 · headline

JevBench Capability Score

Capability ranking of Jev-class systems

Capability Score averages Intelligence and Calibration. kettei-fat leads the Jev-class systems with 86.3.

Jev-class means at most 2× the cost and median latency of Jev (v1.5 reference). How we choose ↘

Adjust cost / latency caps · 2× official
Official
#SystemICCap.$/1k
  1. 1kettei-fatAPIv1.6.43 drawv1.7 · 10 s rule79.857.486.3$0.026*
  2. 2Super DeciderAPIv1.7.0 drawv1.7 · 10 s rule80.056.485.9$0.028
  3. 3Super DeciderAPIv1.7.0 drawv1.7 · 10 s rule78.964.384.9$0.016
  4. 4Fastino GLiDEAPIv1.6.43 drawv1.7 · 10 s rule76.159.184.5$0.023
  5. 5H2O-Lightning-31B v1.0.7Usev1.6.43 drawv1.7 · 10 s rule79.350.684.3$0.044*Use this model
  6. 6Drex MaxAPIv1.6.43 drawv1.7 · 10 s rule78.955.384.2$0.031
  7. 7basal-omni-1.0-ultraUsev1.6.43 drawv1.7 · 10 s rule74.150.482.1$0.045*Use this model
  8. 8DOPEST 1.0 31BUsev1.7.4 drawv1.7 · 10 s rule73.949.282.0$0.049*Use this model
  9. 9SubQ Snap 1.1APIv1.6.43 drawv1.7 · 10 s rule75.860.781.9$0.020*
  10. 10Jeff-1.1-MediumUsev1.7.8 drawv1.7 · 10 s rule74.553.881.9$0.035*Use this model
Show all 202 Jev-class systems (192 more)
  1. 11Arcbright Elf 1 31BUsev1.7.0 drawv1.7 · 10 s rule71.949.781.8$0.047*Use this model
  2. 12Quyet-1.0-LargeUsev1.7 · 10 s rule73.450.581.7$0.045*Use this model
  3. 13Jeff-1.0-LargeUsev1.6.4 drawv1.7 · 10 s rule71.949.281.5$0.049*Use this model
  4. 14SubQ SnapAPIv1.6.43 drawv1.7 · 10 s rule72.357.781.5$0.026*
  5. 15Sieve-31B-v2Usev1.7.5 drawv1.7 · 10 s rule68.750.981.1$0.043*Use this model
  6. 16d1-ultraAPIv1.6.43 drawv1.7 · 10 s rule69.762.380.5$0.018*
  7. 17Bespoke Nimble-v4-maxAPIv1.6.43 drawv1.7 · 10 s rule68.260.779.7$0.020*
  8. 18metask-jev-rain-31B-trained service v3Usev1.6.43 drawv1.7 · 10 s rule72.649.379.6$0.049*Use this model
  9. 19decisio v0.8.0 on gemma-4-31B-itUsev1.7 · 10 s rule70.451.779.6$0.041*Use this model
  10. 20metask-jev-rain-31B-trainedUsev1.6.43 drawv1.7 · 10 s rule70.750.778.9$0.044*Use this model
  11. 21Myles-G 31BAPIv1.6.43 drawv1.7 · 10 s rule73.651.178.8$0.043*
  12. 22Sage 1.3.0APIv1.7 · 10 s rule65.658.278.6$0.025*
  13. 23Bobcat 1.3Usev1.7.0 drawv1.7 · 10 s rule68.149.977.9$0.047*Use this model
  14. 24deck-31BUsev1.7 · 10 s rule73.049.777.6$0.048*Use this model
  15. 25Torchcast Decision 4BUsev1.6.43 drawv1.7 · 10 s rule67.164.877.5$0.015*Use this model
  16. 26Jev 1.13.0APIv1.7 · 10 s rule63.654.777.1$0.032
  17. 27metask-jev-rain-12B-trainedUsev1.6.43 drawv1.7 · 10 s rule66.552.776.9$0.038*Use this model
  18. 28Wald 26B-A4BUsev1.7.0 drawv1.7 · 10 s rule65.549.676.8$0.048*Use this model
  19. 29René-1 31B FP8Usev1.7 · 10 s rule61.746.076.0$0.063*Use this model
  20. 30Decisio 12B tuned commitUsev1.7.6 drawv1.7 · 10 s rule62.858.575.7$0.024*Use this model
  21. 31Jev 1.14.0APIv1.6.43 drawv1.7 · 10 s rule59.954.875.0$0.032*
  22. 32H2O-Lightning-4B v1.1Usev1.6.43 drawv1.7 · 10 s rule60.064.075.0$0.016*Use this model
  23. 33d1APIv1.7 · 10 s rule62.162.874.4$0.017
  24. 34SPX-CD FlashAPIv1.7 · 10 s rule58.452.174.3$0.039*
  25. 35Mercury DecideAPIv1.7 · 10 s rule65.962.173.6$0.018*
  26. 36Celeris-1 DecisionAPIv1.6.43 drawv1.7 · 10 s rule58.960.473.6$0.021*
  27. 37OpenAI DecisionsAPIv1.7 · 10 s rule56.948.673.5$0.052
  28. 38Surogate Rune 26B-A4B v3Usev1.7 · 10 s rule56.149.673.5$0.048*Use this model
  29. 39kettei-chubAPIv1.6.43 drawv1.7 · 10 s rule62.262.773.1$0.018*
  30. 40Bobcat Flash 1.2Usev1.7 · 10 s rule65.649.773.0$0.048*Use this model
  31. 41Torchcast Decision 4BUsev1.6.43 drawv1.7 · 10 s rule60.364.872.6$0.015*Use this model
  32. –Microsoft-Decision-1APIv1.6.43 drawv1.7 · 10 s rule59.262.272.0$0.018PRELIMINARY — API supplement A2 (300 sealed + 300 public), not the S1200 full set; equating pool 4 of 6
  33. 42torchcast-decision-12bUsev1.7 · 10 s rule60.556.471.7$0.028*Use this model
  34. 43decider-12b v2Usev1.7 · 10 s rule61.957.871.3$0.026*Use this model
  35. 44Jev-OmniUsev1.7 · 10 s rule55.556.171.3$0.029*Use this model
  36. 45Winnow-12B Q8Usev1.7 · 10 s rule59.556.671.2$0.028*Use this model
  37. 46Xor 26B-A4BUsev1.7 · 10 s rule57.550.771.2$0.044*Use this model
  38. 47decider-12b v1, stock Gemma-4-12B-itUsev1.7 · 10 s rule59.657.871.0$0.026*Use this model
  39. 48CygnetUsev1.7 · 10 s rule54.856.470.9$0.028*Use this model
  40. 49Microsoft-Decision-1APIv1.7 · 10 s rule57.261.570.8$0.019
  41. 50decisio v0.8.0 on gemma-4-12B-itUsev1.7 · 10 s rule52.359.370.7$0.023*Use this model
  42. 51Myles-G 12BAPIv1.6.43 drawv1.7 · 10 s rule57.399.470.7$0.0011*
  43. 52InstinctAPIv1.7 · 10 s rule47.864.470.5$0.015
  44. 53decider chat on Gemma-4-31B-itUsev1.7 · 10 s rule58.550.870.4$0.044*Use this model
  45. 54baselabs elephant-1Usev1.7.0 drawv1.7 · 10 s rule55.848.470.2$0.052*Use this model
  46. 55GEV-26B-DecideUsev1.7 · 10 s rule49.851.170.1$0.043*Use this model
  47. 56Chon-1.0-12B-FastUsev1.7.1 drawv1.7 · 10 s rule60.358.770.1$0.024*Use this model
  48. 57decisio v0.9.0 on gemma-4-12B-itUsev1.7 · 10 s rule52.359.370.0$0.023*Use this model
  49. 58Blink v0.3 26B-A4BUsev1.6.9 drawv1.7 · 10 s rule49.249.669.9$0.048*Use this model
  50. 59attest-31B-v1.0APIv1.6.43 drawv1.7 · 10 s rule65.250.769.5$0.044*
  51. 60Wald 4B v2.1Usev1.6.9 drawv1.7 · 10 s rule52.046.369.3$0.062*Use this model
  52. 61vLLM-SR d3-miniUsev1.7.0 drawv1.7 · 10 s rule50.864.268.8$0.016*Use this model
  53. 62Torchcast Decision 4BUsev1.6.43 drawv1.7 · 10 s rule54.264.868.7$0.015*Use this model
  54. 63Xevn-4BUsev1.7.0 drawv1.7 · 10 s rule52.166.068.6$0.014*Use this model
  55. 64Chon-1.0-E4BUsev1.7.3 drawv1.7 · 10 s rule55.570.768.5$0.0094*Use this model
  56. 65Hopper 12B trainedUsev1.7 · 10 s rule51.756.268.1$0.029*Use this model
  57. 66SPX-CD-OmniUsev1.7 · 10 s rule48.858.368.1$0.025*Use this model
  58. 67Aplomb 1Usev1.7 · 10 s rule45.257.967.1$0.025*Use this model
  59. 68Bespoke Nimble 9B v3Usev1.7 · 10 s rule56.546.166.8$0.063*Use this model
  60. 69Amazon Strands Decider 26B-A4BUsev1.7.1 drawv1.7 · 10 s rule46.750.566.7$0.045*Use this model
  61. 70Metask rain 4BUsev1.6.4 drawv1.7 · 10 s rule51.151.866.0$0.040*Use this model
  62. 71shisa-de-1Usev1.6.43 drawv1.7 · 10 s rule49.949.865.5$0.047*Use this model
  63. 72Plumb-4BUsev1.7 · 10 s rule42.863.165.3$0.017*Use this model
  64. 73Arcbright Elf 1 12BUsev1.6.43 drawv1.7 · 10 s rule43.857.465.0$0.026*Use this model
  65. 74decider-4b v2Usev1.7 · 10 s rule40.164.564.9$0.015*Use this model
  66. 75Seb-9B-v2Usev1.7.0 drawv1.7 · 10 s rule46.148.964.8$0.051*Use this model
  67. 76HS1 0.1.2APIv1.6.43 drawv1.7 · 10 s rule52.949.164.3$0.050*
  68. 77JevK5 v0.3Usev1.7 · 10 s rule38.563.164.2$0.017*Use this model
  69. 78Bespoke Nimble-v4APIv1.6.43 drawv1.7 · 10 s rule41.460.764.0$0.020*
  70. 79Jeffan-4B-2010Usev1.7.0 drawv1.7 · 10 s rule40.164.164.0$0.016*Use this model
  71. 80Clef-FlashUsev1.7 · 10 s rule42.246.863.8$0.059*Use this model
  72. 81JPT-4BUsev1.6.43 drawv1.7 · 10 s rule40.864.263.8$0.016*Use this model
  73. 82Vansa-3.4APIv1.7 · 10 s rule40.961.963.7$0.019
  74. 83deck-4B v1.0Usev1.7 · 10 s rule38.663.563.6$0.017*Use this model
  75. 84Jebadiah 9B v2.1Usev1.6.43 drawv1.7 · 10 s rule37.248.963.4$0.051*Use this model
  76. 85GLiClass v4 9BUsev1.7.9 drawv1.7 · 10 s rule39.651.263.2$0.042*Use this model
  77. 86Seb-9BUsev1.6.43 drawv1.7 · 10 s rule38.348.963.1$0.051*Use this model
  78. 87janus 4BUsev1.7 · 10 s rule37.664.462.9$0.015*Use this model
  79. 88Amos 1 mediumAPIv1.6.43 drawv1.7 · 10 s rule38.148.762.9$0.051*
  80. 89JevK5 v0.2.0Usev1.7 · 10 s rule36.363.162.8$0.017*Use this model
  81. 90Messier One v0.4Usev1.6.43 drawv1.7 · 10 s rule52.061.162.4$0.020*Use this model
  82. 91Imajev-4BUsev1.7 · 10 s rule34.663.361.9$0.017*Use this model
  83. 92jqvUsev1.7 · 10 s rule33.951.261.9$0.042*Use this model
  84. 93metask-jev-4bUsev1.7 · 10 s rule39.257.761.8$0.026*Use this model
  85. 94Kahn1 4BUsev1.6.43 drawv1.7 · 10 s rule39.750.161.7$0.046*Use this model
  86. 95pplx-decider-flash-v1.2Usev1.6.43 drawv1.7 · 10 s rule45.564.461.7$0.015*Use this model
  87. 96levUsev1.7 · 10 s rule41.161.861.7$0.019*Use this model
  88. 97Decision 2.0 Lux 9BUsev1.6.43 drawv1.7 · 10 s rule35.848.561.6$0.052*Use this model
  89. 98blink-4bUsev1.6.43 drawv1.7 · 10 s rule35.063.561.4$0.016*Use this model
  90. 99HopperUsev1.7 · 10 s rule33.662.361.2$0.018*Use this model
  91. 100Malkuth-4BUsev1.7 · 10 s rule33.855.861.1$0.030*Use this model
  92. 101Decision 4B v1.2Usev1.7 · 10 s rule32.463.160.4$0.017*Use this model
  93. 102jiwo 4bUsev1.6.9 drawv1.7 · 10 s rule34.864.160.2$0.016*Use this model
  94. 103Verdin-4BUsev1.7.0 drawv1.7 · 10 s rule35.462.259.6$0.018*Use this model
  95. 104Wald 4BUsev1.6.4 drawv1.7 · 10 s rule41.054.359.6$0.033*Use this model
  96. 105Quyet-1.0-MediumUsev1.7 · 10 s rule41.763.459.2$0.017*Use this model
  97. 106Decision-4BUsev1.7 · 10 s rule37.965.959.1$0.014*Use this model
  98. 107Manchego v2.1Usev1.7 · 10 s rule32.564.358.3$0.015*Use this model
  99. 108Messier One v0.2Usev1.7 · 10 s rule42.464.858.1$0.015Use this model
  100. 109Diffusion JevUsev1.7 · 10 s rule54.949.658.1$0.048*Use this model
  101. 110jev-localUsev1.7 · 10 s rule41.958.957.5$0.024*Use this model
  102. 111Instinct Dual 4BAPIv1.7 · 10 s rule28.375.057.2$0.0068
  103. 112Decision 4B v1.1Usev1.7 · 10 s rule29.163.157.2$0.017*Use this model
  104. 113JEV Qwen3.5-9B Base NVFP4Usev1.7 · 10 s rule29.347.657.1$0.056*Use this model
  105. 114iota-decide-4bUsev1.6.43 drawv1.7 · 10 s rule29.463.057.1$0.017*Use this model
  106. 115Uniphore-4B-DeciderUsev1.6.43 drawv1.7 · 10 s rule34.264.557.1$0.015*Use this model
  107. 116Jebadiah 9B v2Usev1.6.43 drawv1.7 · 10 s rule30.648.956.9$0.051*Use this model
  108. 117CoCo-Decision-4BUsev1.7 · 10 s rule32.765.256.9$0.014*Use this model
  109. 118APUS-OpenJev-v1-9BUsev1.7 · 10 s rule47.248.756.9$0.051*Use this model
  110. 119Decider-4BUsev1.6.43 drawv1.7 · 10 s rule27.166.256.8$0.013*Use this model
  111. 120typecastlmUsev1.7 · 10 s rule29.864.056.5$0.016*Use this model
  112. 121pjev-2bUsev1.7.0 drawv1.7 · 10 s rule28.863.556.5$0.016*Use this model
  113. 122TypeCastLM 1.4.0Usev1.7 · 10 s rule35.464.856.4$0.015*Use this model
  114. 123RYOTIDE-QwenUsev1.6.43 drawv1.7 · 10 s rule25.262.256.3$0.018*Use this model
  115. 124Compass 0.2.0Usev1.6.43 drawv1.7 · 10 s rule28.957.656.1$0.026*Use this model
  116. 125Standard One 3B SH v4Usev1.7.0 drawv1.7 · 10 s rule34.448.656.0$0.052*Use this model
  117. 126Standard One 3B SHUsev1.6.43 drawv1.7 · 10 s rule32.848.655.6$0.052*Use this model
  118. 127SemIfUsev1.7 · 10 s rule27.163.155.4$0.017*Use this model
  119. 128ArseneLupin v1.1Usev1.6.43 drawv1.7 · 10 s rule28.060.755.1$0.020*Use this model
  120. 129local-jev Qwen3.5-4BUsev1.7 · 10 s rule24.159.355.0$0.023*Use this model
  121. 130Decision 2BUsev1.7 · 10 s rule22.966.454.8$0.013*Use this model
  122. 131Jez-large-1.0.0Usev1.7.0 drawv1.7 · 10 s rule25.670.554.4$0.0096*Use this model
  123. 132NomaUsev1.6.43 drawv1.7 · 10 s rule25.162.054.3$0.019*Use this model
  124. 133spark-s1-4b-v6Usev1.7 · 10 s rule43.260.454.0$0.021*Use this model
  125. 134Imajev-2BUsev1.6.43 drawv1.7 · 10 s rule21.251.854.0$0.041*Use this model
  126. 135RYOTIDE-Qwen9Usev1.6.4 drawv1.7 · 10 s rule27.649.452.5$0.049*Use this model
  127. 136classify-1APIv1.7.3 drawv1.7 · 10 s rule21.850.852.3$0.044
  128. 137Arbiter-4BUsev1.6.43 drawv1.7 · 10 s rule27.466.252.3$0.013*Use this model
  129. 138DecisionNode Edge 0.9.2Usev1.7.0 drawv1.7 · 10 s rule26.871.252.1$0.0091*Use this model
  130. 139system-one-openAPIv1.7 · 10 s rule27.968.351.9$0.011*
  131. 140D1A on Gemma 4 E4BUsev1.7.7 drawv1.7 · 10 s rule19.960.651.1$0.021*Use this model
  132. 141ClassOne Qwen 3.5 9BUsev1.7 · 10 s rule42.249.350.5$0.049*Use this model
  133. 142Liquid AI d1-3BUsev1.6.9 drawv1.7 · 10 s rule22.150.649.5$0.044*Use this model
  134. 143ZeroEntropy zerank-2Usev1.7 · 10 s rule15.648.449.4$0.052*Use this model
  135. 144open-alternative-jevUsev1.7 · 10 s rule22.363.349.1$0.017*Use this model
  136. 145OpenSourceJevUsev1.7 · 10 s rule23.269.148.6$0.011*Use this model
  137. 146EXAONE-4.0-1.2B-JEV v0.3Usev1.7 · 10 s rule14.857.947.7$0.025*Use this model
  138. 147Malkuth-2BUsev1.7 · 10 s rule15.965.647.6$0.014*Use this model
  139. 148DecisionNode Edge Mini 0.9.2Usev1.7.0 drawv1.7 · 10 s rule17.680.247.6$0.0046*Use this model
  140. 149Nemotron Diffusion 8BUsev1.7 · 10 s rule20.455.547.5$0.030*Use this model
  141. 150jiwo 0.8bUsev1.6.9 drawv1.7 · 10 s rule15.978.447.2$0.0052*Use this model
  142. 151decision-machine-1APIv1.7 · 10 s rule13.158.346.1$0.024
  143. 152decisor-4bUsev1.6.4 drawv1.7 · 10 s rule39.664.045.7$0.016*Use this model
  144. 153D1A on Gemma 4 E2BUsev1.7.7 drawv1.7 · 10 s rule8.960.645.5$0.021*Use this model
  145. 154Qwen3-Reranker-4BUsev1.7 · 10 s rule16.348.444.7$0.052*Use this model
  146. 155decider-2bUsev1.7 · 10 s rule20.764.944.6$0.015*Use this model
  147. 156BAAI bge-reranker-v2-m3Usev1.7 · 10 s rule0.159.344.1$0.023*Use this model
  148. 157kev 4BUsev1.7 · 10 s rule19.565.844.0$0.014*Use this model
  149. 158Mixedbread mxbai-rerank-base-v2Usev1.7 · 10 s rule1.160.343.9$0.021*Use this model
  150. 159lev-350mUsev1.7 · 10 s rule4.880.043.3$0.0046*Use this model
  151. 160Jez-small-1.0.0Usev1.7.0 drawv1.7 · 10 s rule4.679.743.2$0.0048*Use this model
  152. 161Certo v1Usev1.7 · 10 s rule0.296.943.2$0.0013*Use this model
  153. 162PurpleMIST-Mini-1.0Usev1.7.0 drawv1.7 · 10 s rule8.470.843.0$0.0094*Use this model
  154. 163Chon-1.0-12BUsev1.6.43 drawv1.7 · 10 s rule27.349.043.0$0.050*Use this model
  155. 164smalljev semantic-v9Usev1.7 · 10 s rule8.760.843.0$0.020*Use this model
  156. 165Vega-UUsev1.7.0 drawv1.7 · 10 s rule8.564.942.6$0.015*Use this model
  157. 166jul fastUsev1.7 · 10 s rule7.569.042.6$0.011*Use this model
  158. 167Krino-2BUsev1.6.43 drawv1.7 · 10 s rule22.061.042.4$0.020*Use this model
  159. 168Alibaba GTE Reranker ModernBERT-baseUsev1.7 · 10 s rule0.069.341.7$0.011*Use this model
  160. 169Jeetu Decider 2BUsev1.7.2 drawv1.7 · 10 s rule10.270.341.7$0.0098*Use this model
  161. 170watt-flash-0.1Usev1.7 · 10 s rule3.889.041.6$0.0023Use this model
  162. 171Decision FastUsev1.7 · 10 s rule6.180.140.6$0.0046*Use this model
  163. 172openJev Verdict 1.4Usev1.7 · 10 s rule5.086.640.4$0.0028*Use this model
  164. 173Quyet-1.0-Small-ENUsev1.7 · 10 s rule6.189.139.5$0.0023*Use this model
  165. 174ClassOne Gemma 4 E4BUsev1.6.9 drawv1.7 · 10 s rule28.269.338.4$0.011*Use this model
  166. 175ClassOne Qwen 3.5 4BUsev1.6.9 drawv1.7 · 10 s rule27.365.038.1$0.015*Use this model
  167. 176OpenDecisionUsev1.7 · 10 s rule7.479.138.0$0.0050*Use this model
  168. 177verdict-smallUsev1.7 · 10 s rule2.3100.037.6$0.00087*Use this model
  169. 178kev 0.6BUsev1.7 · 10 s rule7.680.136.9$0.0046*Use this model
  170. 179Vega 4BUsev1.6.9 drawv1.7 · 10 s rule15.747.236.6$0.058*Use this model
  171. 180WaterSheepUsev1.7 · 10 s rule5.368.735.5$0.011*Use this model
  172. 181Raw Phi-4 mini direct logitsUsev1.7 · 10 s rule13.753.235.3$0.036*Use this model
  173. 182Fastino GLiNER-2.5-DecideAPIv1.7 · 10 s rule7.045.834.6$0.064
  174. 183Raw Qwen3 4B Instruct 2507 direct logitsUsev1.7 · 10 s rule34.063.434.6$0.017*Use this model
  175. 184Vega 0.8BUsev1.6.9 drawv1.7 · 10 s rule3.161.533.0$0.019*Use this model
  176. 185kev 0.5BUsev1.7 · 10 s rule4.680.132.9$0.0046*Use this model
  177. 186Liquid AI d1-omni-600MUsev1.6.9 drawv1.7 · 10 s rule3.373.132.8$0.0079*Use this model
  178. 187Quyet-1.0-SmallUsev1.7 · 10 s rule4.079.632.5$0.0048*Use this model
  179. 188Quyet-1.0-TinyUsev1.7 · 10 s rule3.788.431.1$0.0024*Use this model
  180. 189Open Jev JSON CanvasUsev1.7 · 10 s rule61.249.330.6$0.049*Use this model
  181. 190Phocinae-Largha-150M-v1Usev1.6.43 drawv1.7 · 10 s rule3.695.330.2$0.0014*Use this model
  182. 191JevActAPIv1.7 · 10 s rule10.168.628.8$0.011*
  183. 192openJev VerdictUsev1.7 · 10 s rule4.186.626.6$0.0028*Use this model
  184. 193Tacet SonataUsev1.7 · 10 s rule2.289.024.2$0.0023*Use this model
  185. 194CLM-8BUsev1.7 · 10 s rule0.150.520.1$0.045*Use this model
  186. 195ClassOne Qwen 3.5 2BUsev1.6.9 drawv1.7 · 10 s rule5.870.219.2$0.0098*Use this model
  187. 196Deem 0.8B v1Usev1.7 · 10 s rule6.580.318.1$0.0045*Use this model
  188. 197Laya multilingualUsev1.7 · 10 s rule0.382.217.0$0.0039*Use this model
  189. 198ClassOne Gemma 4 E2BUsev1.7 · 10 s rule7.870.215.2$0.0098*Use this model
  190. 199Julia-1Usev1.6.43 drawv1.7 · 10 s rule2.489.79.9$0.0022*Use this model
  191. 200Raw Qwen3 1.7B direct logitsUsev1.7 · 10 s rule5.168.58.3$0.011*Use this model
  192. 201Raw Qwen3 0.6B direct logitsUsev1.7 · 10 s rule5.677.68.2$0.0056*Use this model

Wide coloured bar = Capability Score (0–100). Thin bars: cost above, median latency below; shared log ratio scale 0.25× … 64× Jev (v1.5 reference), ticks at 1× and 2× (cap). Shorter is cheaper or faster. * = est. (estimated cost). # counts ranked Jev-class systems; “–” marks unranked or outside systems. Tap ⓘ or hover a row or thin bar for details.

Cost and latency correlate here (Spearman ρ = 0.28, n = 282).

  • Jev — reference (TypeSafe, closed) (2)
  • Closed API (weights not public) (32)
  • Open weights · LLM decoder (208)
  • Open weights · diffusion LM (6)
  • Open weights · encoder / classifier (21)
  • Open weights · reranker (5)
  • Base model control (no decision fine-tune, raw logits) (5)
  • System (router / cascade / ensemble) (3)
  • green: ≤ reference
  • amber: ≤ cap (2× reference)
  • red: > cap

Use = run this open-weights model yourself: own endpoint, API, local install or hardware, on Decision Models. Decision Models is run by the same company as Benchmark Heaven. Every open-weights row gets the same link, whether or not we serve the model; it never affects scores or ranks. Independence

Show general-purpose LLMs and other systems outside the limits

Sorted by Capability Score, not numbered. Each row says which limit it misses, measured against Jev ($0.032 per 1,000 decisions, median 0.62 s).

  1. –GPT-6 Luna (medium)APIv1.7 · 10 s rule96.938.498.4$0.11Outside: cost 3.5× Jev (v1.5 reference), latency 3.1× Jev (v1.5 reference)
  2. –GPT-6 Luna (low)APIv1.7 · 10 s rule95.739.097.1$0.11Outside: cost 3.3× Jev (v1.5 reference), latency 3.1× Jev (v1.5 reference)
  3. –GPT-5.6 LunaAPIv1.7 · 10 s rule93.030.293.6$0.21Outside: cost 6.6× Jev (v1.5 reference), latency 2.6× Jev (v1.5 reference)
  4. –DeepSeek V4.1 FlashAPIv1.7 · 10 s rule84.519.689.6$0.48Outside: cost 14.9× Jev (v1.5 reference), latency 3.6× Jev (v1.5 reference)
  5. –baselabs mammoth-1Usev1.7.0 drawv1.7 · 10 s rule76.239.585.2$0.10*Use this modelOutside: cost 3.2× Jev (v1.5 reference)
  6. –KnowLine 31B-Gen1Usev1.7.0 drawv1.7 · 10 s rule77.440.985.2$0.093*Use this modelOutside: cost 2.9× Jev (v1.5 reference)
  7. –Fastino GLiDE 1.2Usev1.7.0 drawv1.7 · 10 s rule76.430.684.9$0.21*Use this modelOutside: cost 6.4× Jev (v1.5 reference)
  8. –Perplexity Decider v1.1 27BUsev1.7 · 10 s rule75.629.982.7$0.22*Use this modelOutside: cost 6.7× Jev (v1.5 reference)
  9. –vLLM-SR d3-turboUsev1.6.43 drawv1.7 · 10 s rule74.829.882.6$0.22*Use this modelOutside: cost 6.8× Jev (v1.5 reference)
  10. –StartLux-Decision v1.1 27B InstantUsev1.6.43 drawv1.7 · 10 s rule73.230.482.6$0.21*Use this modelOutside: cost 6.5× Jev (v1.5 reference)
  11. –vLLM-SR d3Usev1.6.43 drawv1.7 · 10 s rule72.929.882.1$0.22*Use this modelOutside: cost 6.8× Jev (v1.5 reference)
  12. –Jeffan-31b-1710Usev1.7.0 drawv1.7 · 10 s rule71.642.881.7$0.081*Use this modelOutside: cost 2.5× Jev (v1.5 reference)
  13. –AJev lm6qUsev1.7.0 drawv1.7 · 10 s rule71.541.081.4$0.093*Use this modelOutside: cost 2.9× Jev (v1.5 reference)
  14. –Nimble-v4-maxUsev1.7.0 drawv1.7 · 10 s rule71.030.181.2$0.21*Use this modelOutside: cost 6.6× Jev (v1.5 reference)
  15. –Turion Jev 31BUsev1.7.1 drawv1.7 · 10 s rule70.940.680.9$0.096*Use this modelOutside: cost 3.0× Jev (v1.5 reference)
  16. –pplx-decider-v1.2Usev1.6.43 drawv1.7 · 10 s rule71.730.080.8$0.21*Use this modelOutside: cost 6.6× Jev (v1.5 reference)
  17. –torchcast-decision-27bUsev1.7 · 10 s rule70.430.380.2$0.21*Use this modelOutside: cost 6.5× Jev (v1.5 reference)
  18. –Vera LargeUsev1.7.0 drawv1.7 · 10 s rule67.829.979.6$0.22*Use this modelOutside: cost 6.7× Jev (v1.5 reference)
  19. –Kev 27BUsev1.7 · 10 s rule64.218.378.9$0.53*Use this modelOutside: cost 16.4× Jev (v1.5 reference)
  20. –StartLux-Decision v1.1 27B AutoThinkUsev1.6.43 drawv1.7 · 10 s rule66.723.578.8$0.35*Use this modelOutside: cost 11.0× Jev (v1.5 reference)
  21. –SPX-CD ProAPIv1.7 · 10 s rule64.643.178.5$0.079*Outside: cost 2.4× Jev (v1.5 reference)
  22. –Amos 1 largeAPIv1.6.43 drawv1.7 · 10 s rule68.230.078.0$0.21*Outside: cost 6.6× Jev (v1.5 reference)
  23. –Eikos-27BUsev1.7 · 10 s rule64.428.777.2$0.24*Use this modelOutside: cost 7.4× Jev (v1.5 reference)
  24. –Fastino GLiDEAPIv1.7 · 10 s rule66.135.977.0$0.14Outside: cost 4.3× Jev (v1.5 reference)
  25. –wity-1APIv1.7 · 10 s rule67.958.876.9$0.024Outside: latency 3.0× Jev (v1.5 reference)API price · eligibility checked at the developer's list price; at base-model pricing it would exceed the cost cap (2.70× Jev)
  26. –Aleria Judge 1.1.1Usev1.7.0 drawv1.7 · 10 s rule68.035.876.9$0.14*Use this modelOutside: cost 4.3× Jev (v1.5 reference)
  27. –KnowLine 35B-A3B-Gen1.5Usev1.7.0 drawv1.7 · 10 s rule60.324.776.7$0.32*Use this modelOutside: cost 10.0× Jev (v1.5 reference)
  28. –OpenJevUsev1.7 · 10 s rule70.228.575.8$0.24*Use this modelOutside: cost 7.5× Jev (v1.5 reference), latency 3.1× Jev (v1.5 reference)
  29. –ClefUsev1.7 · 10 s rule59.128.175.3$0.25*Use this modelOutside: cost 7.7× Jev (v1.5 reference)
  30. –SimpleJev Qwen3.8-27BUsev1.7 · 10 s rule57.915.174.9$0.68*Use this modelOutside: cost 20.9× Jev (v1.5 reference), latency 2.3× Jev (v1.5 reference)
  31. –JEV-27BUsev1.7 · 10 s rule56.831.073.9$0.20*Use this modelOutside: cost 6.2× Jev (v1.5 reference)
  32. –Nagi-ENORMOUS 27BUsev1.6.43 drawv1.7 · 10 s rule62.430.073.8$0.22*Use this modelOutside: cost 6.7× Jev (v1.5 reference)
  33. –RSI-Jev v6.1-VL 27BUsev1.6.9 drawv1.7 · 10 s rule61.631.173.5$0.20*Use this modelOutside: cost 6.1× Jev (v1.5 reference)
  34. –Sage 1.4APIv1.6.43 drawv1.7 · 10 s rule63.759.573.3$0.022*Outside: latency 2.8× Jev (v1.5 reference)
  35. –Autoloops – Gemma 4 31B ITAPIv1.7 · 10 s rule58.140.572.7$0.096Outside: cost 3.0× Jev (v1.5 reference)
  36. –swanOneUsev1.7 · 10 s rule56.242.272.4$0.085*Use this modelOutside: cost 2.6× Jev (v1.5 reference)
  37. –FrontiersMind Lumma-fev-27BUsev1.7.0 drawv1.7 · 10 s rule55.517.972.3$0.54*Use this modelOutside: cost 16.9× Jev (v1.5 reference)
  38. –AutoJev-27BUsev1.7 · 10 s rule55.629.471.6$0.23*Use this modelOutside: cost 7.0× Jev (v1.5 reference)
  39. –SimpleJev Qwen3.8-27BAPIv1.7 · 10 s rule52.914.971.6$0.69*Outside: cost 21.3× Jev (v1.5 reference), latency 2.1× Jev (v1.5 reference)
  40. –Decision 2.0 Vega 27BUsev1.7 · 10 s rule60.429.671.3$0.22*Use this modelOutside: cost 6.9× Jev (v1.5 reference)
  41. –JPT-35B-A3BUsev1.7 · 10 s rule51.433.670.7$0.16*Use this modelOutside: cost 5.0× Jev (v1.5 reference)
  42. –Jebadiah 27BUsev1.7 · 10 s rule57.529.970.2$0.22*Use this modelOutside: cost 6.7× Jev (v1.5 reference)
  43. –NInfer Qwen3.8-Flash-Next mixedUsev1.7 · 10 s rule49.042.569.7$0.082*Use this modelOutside: cost 2.5× Jev (v1.5 reference)
  44. –Respan Span-02Usev1.6.43 drawv1.7 · 10 s rule60.945.169.3$0.068*Use this modelOutside: cost 2.1× Jev (v1.5 reference)
  45. –JevOneUsev1.7 · 10 s rule46.039.868.9$0.10*Use this modelOutside: cost 3.1× Jev (v1.5 reference)
  46. –SimpleJev Qwen3.6-35B-A3BAPIv1.7 · 10 s rule56.335.168.3$0.15*Outside: cost 4.5× Jev (v1.5 reference), latency 2.1× Jev (v1.5 reference)
  47. –NInfer Qwen3.8-27B NVFP4Usev1.7 · 10 s rule51.829.168.0$0.23*Use this modelOutside: cost 7.1× Jev (v1.5 reference)
  48. –Open-Jev 27B v1.1Usev1.7 · 10 s rule52.514.468.0$0.72*Use this modelOutside: cost 22.2× Jev (v1.5 reference), latency 2.9× Jev (v1.5 reference)
  49. –LitJevUsev1.7 · 10 s rule43.328.466.6$0.24*Use this modelOutside: cost 7.6× Jev (v1.5 reference), latency 5.1× Jev (v1.5 reference)
  50. –Tern-35B-A3BUsev1.7.0 drawv1.7 · 10 s rule44.142.166.4$0.085*Use this modelOutside: cost 2.6× Jev (v1.5 reference)
  51. –decider-35b-a3bUsev1.7 · 10 s rule48.734.466.2$0.15*Use this modelOutside: cost 4.8× Jev (v1.5 reference)
  52. –Bev / Bonsai 27BUsev1.7 · 10 s rule43.828.266.1$0.25*Use this modelOutside: cost 7.6× Jev (v1.5 reference), latency 3.5× Jev (v1.5 reference)
  53. –Imajev-9BUsev1.6.43 drawv1.7 · 10 s rule36.730.862.4$0.20*Use this modelOutside: cost 6.3× Jev (v1.5 reference)
  54. –Open-Jev 9BUsev1.7 · 10 s rule43.933.161.8$0.17*Use this modelOutside: cost 5.3× Jev (v1.5 reference), latency 2.3× Jev (v1.5 reference)
  55. –JEB-35B-A3BUsev1.6.43 drawv1.7 · 10 s rule33.242.961.6$0.080*Use this modelOutside: cost 2.5× Jev (v1.5 reference)
  56. –Qwen3.5-9B Jev-like data-mix v2Usev1.7 · 10 s rule41.945.760.9$0.065*Use this modelOutside: cost 2.001× Jev (v1.5 reference)
  57. –Standard One 8B SHUsev1.6.9 drawv1.7 · 10 s rule34.543.260.0$0.078*Use this modelOutside: cost 2.4× Jev (v1.5 reference)
  58. –Gemini 3.1 Flash-LiteAPIv1.7 · 10 s rule58.629.259.8$0.23Outside: cost 7.1× Jev (v1.5 reference)
  59. –reflex 4BUsev1.7 · 10 s rule30.863.158.9$0.017*Use this modelOutside: latency 4.7× Jev (v1.5 reference)
  60. –openjev-sglangAPIv1.7 · 10 s rule38.835.658.7$0.14*Outside: cost 4.3× Jev (v1.5 reference), latency 2.1× Jev (v1.5 reference)
  61. –Standard One 8BUsev1.7 · 10 s rule32.843.358.4$0.078*Use this modelOutside: cost 2.4× Jev (v1.5 reference)
  62. –Clef-omniUsev1.6.9 drawv1.7 · 10 s rule34.526.858.3$0.28*Use this modelOutside: cost 8.5× Jev (v1.5 reference)
  63. –Bespoke Nimble 9BUsev1.7 · 10 s rule43.136.856.5$0.13*Use this modelOutside: cost 4.0× Jev (v1.5 reference)
  64. –kev 8BUsev1.7 · 10 s rule29.340.450.0$0.097*Use this modelOutside: cost 3.0× Jev (v1.5 reference)
  65. –Nev-2BUsev1.6.43 drawv1.6 · re-measure pending20.271.049.9$0.0092*Use this modelOutside: latency 20.8× Jev (v1.5 reference)
  66. –JADEUsev1.7 · 10 s rule47.933.649.2$0.16*Use this modelOutside: cost 5.1× Jev (v1.5 reference)
  67. –Qwen3.8 27BAPIv1.7 · 10 s rule33.40.048.6$2.18*Outside: cost 67.4× Jev (v1.5 reference), latency 13.5× Jev (v1.5 reference)
  68. –Open-Jev 2BUsev1.7 · 10 s rule21.833.147.7$0.17*Use this modelOutside: cost 5.3× Jev (v1.5 reference)
  69. –Laya typed-decisionsUsev1.7 · 10 s rule5.582.544.2$0.0038*Use this modelOutside: latency 3.5× Jev (v1.5 reference)
  70. –Qwen3.5-0.8B Decision ModelUsev1.6 · re-measure pending7.379.542.6$0.0048*Use this modelOutside: latency 18.3× Jev (v1.5 reference)
  71. –Gutsy 0.8B v0.3Usev1.6.9 drawv1.7 · 10 s rule8.678.442.2$0.0053*Use this modelOutside: latency 3.1× Jev (v1.5 reference)
  72. –Reflex Instinct 0.6BUsev1.6.43 drawv1.6 · re-measure pending4.079.838.6$0.0047*Use this modelOutside: latency 6.3× Jev (v1.5 reference)
  73. –Vega-UUsev1.6.43 drawv1.7 · 10 s rule5.379.236.1$0.0049*Use this modelOutside: latency 2.6× Jev (v1.5 reference)
  74. –open-jev-deberta-v3-largeUsev1.7 · 10 s rule2.877.635.4$0.0056*Use this modelOutside: latency 5.7× Jev (v1.5 reference)
  75. –modernbert-jev v2Usev1.6.9 drawv1.6 · re-measure pending2.670.434.6$0.0097*Use this modelOutside: latency 8.7× Jev (v1.5 reference)
  76. –LayaUsev1.7 · 10 s rule1.884.932.5$0.0032*Use this modelOutside: latency 3.3× Jev (v1.5 reference)
  77. –SimpleJevUsev1.7 · 10 s rule12.770.331.1$0.0098*Use this modelOutside: latency 14.9× Jev (v1.5 reference)
  78. –Raw Qwen3 8B direct logitsUsev1.7 · 10 s rule26.545.530.1$0.065*Use this modelOutside: cost 2.02× Jev (v1.5 reference)
  79. –system-oneUsev1.7 · 10 s rule29.345.029.7$0.068*Use this modelOutside: cost 2.1× Jev (v1.5 reference)
  80. –MirrorUsev1.7 · 10 s rule0.089.325.8$0.0023*Use this modelOutside: latency 2.1× Jev (v1.5 reference)

Jev-class = cost per decision at most 2× Jev's (≤ $0.065 per 1,000 decisions) and median latency at most 2× Jev's (≤ 1.23 s), the adjusted p50 — the same median the speed chart plots, not the four-axis Speed score. 202 of 282 systems qualify; the other 80, including the general-purpose LLMs, are listed below the divider in the ranking. The charts below show speed and cost beside Capability Score; the official JevBench Score weighs all four axes.

Capability against cost and speed

Jev-class systems are shown by default. Bubble size follows the official JevBench Score; official rank stays unchanged. The five most capable Jev-class systems are labelled.

Capability vs cost

Upper right is best: more capable and cheaper. Cost is USD per 1,000 decisions on a log scale. The dashed line is 2× Jev’s cost.

Capability vs cost: 201 systems. Upper right is best: more capable and cheaper.102030405060708090100$0.0010$0.010$0.10$ per 1,000 decisions (log)Capability↑2× Jev (v1.5 reference)← priciercheaper →1. kettei-fat2. Super Decider3. Super Decider4. Fastino GLiDE5. H2O-Lightning-31B v1.0.7
201 systems. Tap a bubble for its values.

Capability vs speed

Upper right is best: more capable and faster. Speed here is the median-latency speed — the same adjusted median (p50) latency the Jev-class limit uses — a log scale, so each 20 points is 10× faster (median under the numbers). Self-hosted systems are plotted at the est. incl. network latency (2x measured + 150 ms); their measured loopback p50 (on-GPU, no network; not directly comparable with API latency) is in the tooltip. The dashed line is 2× Jev’s median latency.

Capability vs speed: 201 systems. Upper right is best: more capable and faster.010203040506070809010070≈3.2 s80≈1.0 s90≈316 ms100≈100 msMedian-latency speedCapability↑2× Jev (v1.5 reference)← slowerfaster →1. kettei-fat2. Super Decider3. Super Decider4. Fastino GLiDE5. H2O-Lightning-31B v1.0.7
201 systems. Tap a bubble for its values.
  • Jev — reference (TypeSafe, closed) (2)
  • Closed API (weights not public) (31)
  • Open weights · LLM decoder (208)
  • Open weights · diffusion LM (6)
  • Open weights · encoder / classifier (21)
  • Open weights · reranker (5)
  • Base model control (no decision fine-tune, raw logits) (5)
  • System (router / cascade / ensemble) (3)
  • faint = outside Jev-class
313 of 313 systems

Model kind

Jev-class

Release

Parameters

No system in this release reports an exact parameter count.

Developer/API price $/1k
Base-model price $/1k
Scored cost $/1k
Alternative price $/1k
p50 latency (s)
p95 latency (s)

JevBench v1.7.9

JevBench Composite Score: 277 ranked systems

Official· four axes 0–100, equal-weight harmonic mean · Method notes ↓

Weights:
Adjust weights ↓

View by:Capability ↑

The official order weighs Intelligence, Calibration, Speed and Cost equally. Each button re-sorts the same systems by one axis.

Greener = stronger within its column.

282 of 282 systems, sorted by official rank, #1 first.

  1. 1Super DeciderAPIfine-tune · 26B-A4B · NVFP4Base model: google/diffusiongemma-26B-A4B-itmodel card*178.5I 79C 91S 85K 64$0.016
  2. 2Myles-G 12BAPIClosed APIBase model: google/gemma-4-12B-itmodel card*278.2I 57C 84S 85K 99est.$0.0011
  3. 3kettei-fatAPIClosed APIBase model: undisclosed77.7I 80C 93S 93K 57est.$0.026
  4. 4Fastino GLiDE (hosted API, 22 Oct 2026)APIClosed APIBase model: undisclosed*376.4I 76C 93S 87K 59$0.023
  5. 5Torchcast Decision 4B (v1.3)Usefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource76.2I 67C 88S 92K 65est.$0.015
  6. 6d1-ultraAPIClosed APIBase model: undisclosed76.0I 70C 91S 89K 62est.$0.018
  7. 7SubQ Snap 1.1APIClosed APIBase model: undisclosed75.7I 76C 88S 85K 61est.$0.020
  8. 8Drex MaxAPIClosed APIBase model: Qwen3.8-27Bvendor statement*475.4I 79C 90S 90K 55$0.031
  9. 9Bespoke Nimble-v4-maxAPIClosed APIBase model: Qwen/Qwen3.8-27Bmodel card*574.8I 68C 91S 88K 61est.$0.020
  10. 10Super Decider (thinking)APIfine-tune · 26B-A4B · NVFP4Base model: google/diffusiongemma-26B-A4B-itmodel card*674.5I 80C 92S 80K 56$0.028
  11. 11SubQ SnapAPIClosed APIBase model: undisclosed74.4I 72C 91S 86K 58est.$0.026
  12. 12H2O-Lightning-4B v1.1Usefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource73.8I 60C 90S 92K 64est.$0.016
  13. 13Sage 1.3.0APIClosed APIBase model: undisclosed73.7I 66C 92S 92K 58est.$0.025
  14. 14kettei-chubAPIClosed APIBase model: undisclosed73.4I 62C 84S 95K 63est.$0.018
  15. 15H2O-Lightning-31B v1.0.7Usefine-tune · 31BBase model: google/gemma-4-31B-itsource73.1I 79C 89S 90K 51est.$0.044
  16. 16Torchcast Decision 4B (v1.2)Usefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource73.0I 60C 85S 91K 65est.$0.015
  17. 17d1APIClosed APIBase model: undisclosed72.8I 62C 87S 88K 63$0.017
  18. 18Mercury DecideAPISystemBase model: undisclosedsource72.2I 66C 81S 85K 62est.$0.018
  19. 19Decisio 12B tuned commitUsemerge · 12B · BF16Base model: google/gemma-4-12B-itsource72.2I 63C 89S 90K 58est.$0.024
  20. 20Chon-1.0-E4BUsefine-tune · 4BBase model: google/gemma-4-E4B-itsource71.8I 55C 82S 89K 71est.$0.0094
Show all 282 systems (257 more ranked, 5 more not ranked)
  1. 21decisio v0.8.0 on gemma-4-31B-itUse31B · FP8 on loadBase model: google/gemma-4-31B-itsource71.7I 70C 89S 91K 52est.$0.041
  2. 22basal-omni-1.0-ultraUsemerge · 31BBase model: google/gemma-4-31B-itsource71.5I 74C 90S 87K 50est.$0.045
  3. 23Jev 1.13.0APIJev referenceBase model: undisclosed71.4I 64C 91S 91K 55$0.032
  4. Microsoft-Decision-1 (Azure Foundry) — A2 re-run (preliminary)APIPRELIMINARY — API supplement A2 (300 sealed + 300 public), not the S1200 full set; equating pool 4 of 6Closed APIBase model: Qwen3.5-9Bvendor statementvendor statement · Base model disclosed by Microsoft; the Decision-1 endpoint is evaluated as a hosted API offering.71.4I 59C 85S 89K 62$0.018
  5. 24Quyet-1.0-LargeUseadapter · 31BBase model: google/gemma-4-31B-itsource71.3I 73C 90S 86K 51est.$0.045
  6. 25Celeris-1 DecisionAPIClosed APIBase model: undisclosed71.0I 59C 88S 87K 60est.$0.021
  7. 26Sieve-31B-v2Usefine-tune · 31B · BF16Base model: google/gemma-4-31B-itsource70.7I 69C 93S 86K 51est.$0.043
  8. 27Torchcast Decision 4B (v1.1)Usefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource70.7I 54C 83S 94K 65est.$0.015
  9. 28Myles-G 31BAPIClosed APIBase model: google/gemma-4-31B-itmodel card*770.6I 74C 84S 86K 51est.$0.043
  10. 29Arcbright Elf 1 31BUsefine-tune · 31B · bf16Base model: google/gemma-4-31B-itsource70.5I 72C 92S 90K 50est.$0.047
  11. 30Jeff-1.1-MediumUsefine-tune · 12B · BF16Base model: google/gemma-4-12B-itsource70.5I 74C 89S 74K 54est.$0.035
  12. 31decider-12b v2Usefine-tune · 12BBase model: google/gemma-4-12B-itsource70.3I 62C 81S 90K 58est.$0.026
  13. 32Xevn-4BUsefine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4Bsource70.2I 52C 85S 92K 66est.$0.014
  14. 33Jev 1.14.0APIJev referenceBase model: undisclosed70.2I 60C 90S 91K 55est.$0.032
  15. 34torchcast-decision-12bUseadapter · 12BBase model: google/gemma-4-12B-itsource69.8I 61C 83S 91K 56est.$0.028
  16. 35decider-12b v1, stock Gemma-4-12B-itUse12BBase model: google/gemma-4-12B-itsource69.8I 60C 83S 90K 58est.$0.026
  17. 36metask-jev-rain-31B-trainedUseadapter · 31B · W4A16 INT4Base model: google/gemma-4-31B-itsource69.6I 71C 87S 82K 51est.$0.044
  18. 37vLLM-SR d3-miniUsefine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4Bsource69.5I 51C 87S 93K 64est.$0.016
  19. 38Bobcat 1.3Usefine-tune · 31B · fp8Base model: google/gemma-4-31B-itsource69.4I 68C 88S 89K 50est.$0.047
  20. 39Microsoft-Decision-1 (Azure Foundry)APIClosed APIBase model: Qwen3.5-9Bvendor statementvendor statement*869.1I 57C 84S 81K 61$0.019
  21. 40wity-1APIClosed APIBase model: undisclosedBase-model reference price (base undisclosed at the author's request): 43.0 (would be #91)69.0I 68C 86S 68K 59$0.024
  22. 41Chon-1.0-12B-FastUseadapter · 12B · bf16Base model: google/gemma-4-12B-itsource68.9I 60C 80S 84K 59est.$0.024
  23. 42Winnow-12B Q8Usemerge · 12B · GGUF Q8_0Base model: google/gemma-4-12B-itsource68.8I 60C 83S 86K 57est.$0.028
  24. 43deck-31BUse31B · FP8Base model: google/gemma-4-31B-itsource68.6I 73C 82S 86K 50est.$0.048
  25. 44Jeff-1.0-LargeUsefine-tune · 31BBase model: google/gemma-4-31B-itsource68.5I 72C 91S 88K 49est.$0.049
  26. 45CygnetUse12BBase model: google/gemma-4-12B-itsource68.5I 55C 87S 91K 56est.$0.028
  27. 46Wald 26B-A4BUsefine-tune · 26B-A4B · bf16Base model: google/gemma-4-26B-A4B-itsource68.2I 66C 88S 92K 50est.$0.048
  28. 47decisio v0.8.0 on gemma-4-12B-itUse12BBase model: google/gemma-4-12B-itsource68.1I 52C 89S 86K 59est.$0.023
  29. 48attest-31B-v1.0APIbase · 31BBase model: undisclosed67.9I 65C 74S 96K 51est.$0.044
  30. 49DOPEST 1.0 31BUsefine-tune · 31B · BF16Base model: google/gemma-4-31B-itsource67.8I 74C 90S 82K 49est.$0.049
  31. 50Sage 1.4APIClosed APIBase model: undisclosed67.7I 64C 83S 69K 59est.$0.022
  32. 51decisio v0.9.0 on gemma-4-12B-itUse12BBase model: google/gemma-4-12B-itsource67.7I 52C 88S 85K 59est.$0.023
  33. 52Jev-OmniUsefine-tuneBase model: google/gemma-4-12B-itsource67.6I 56C 87S 85K 56est.$0.029
  34. 53metask-jev-rain-12B-trainedUseadapter · 12B · W4A16 INT4Base model: google/gemma-4-12B-it-qat-q4_0-unquantizedsource67.3I 66C 87S 72K 53est.$0.038
  35. 54Bobcat Flash 1.2Usefine-tune · 26B / 4B active · FP8 on loadBase model: google/gemma-4-26B-A4B-itsource67.1I 66C 81S 91K 50est.$0.048
  36. 55Xor 26B-A4BUsemerge · 26B-A4BBase model: google/gemma-4-26B-A4B-itsource66.8I 57C 85S 91K 51est.$0.044
  37. 56SPX-CD FlashAPISystemBase model: Qwen/Qwen3.6-35B-A3Bmodel card*966.6I 58C 90S 79K 52est.$0.039
  38. 57Messier One v0.4Usefine-tune · 4BBase model: undisclosed66.1I 52C 73S 89K 61est.$0.020
  39. 58decider chat on Gemma-4-31B-itUse31BBase model: google/gemma-4-31B-itsource66.0I 59C 82S 86K 51est.$0.044
  40. 59Hopper 12B trainedUsefine-tune · 12BBase model: google/gemma-4-12B-itsource65.6I 52C 85S 84K 56est.$0.029
  41. 60metask-jev-rain-31B-trained service v3Useadapter · 31B · W4A16 INT4Base model: google/gemma-4-31B-itsource65.6I 73C 87S 73K 49est.$0.049
  42. 61Surogate Rune 26B-A4B v3Use26B-A4BBase model: google/gemma-4-26B-A4B-itsource*1065.0I 56C 91S 87K 50est.$0.048
  43. 62GEV-26B-DecideUsedistilled · 26B-A4BBase model: google/gemma-4-26B-A4B-itsource64.2I 50C 90S 90K 51est.$0.043
  44. 63SPX-CD-OmniUsefine-tune · 12BBase model: google/gemma-4-12B-itsource63.1I 49C 87S 89K 58est.$0.025
  45. 64Metask rain 4BUsefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource62.5I 51C 81S 78K 52est.$0.040
  46. 65shisa-de-1Usefine-tune · 26B-A4BBase model: google/gemma-4-26B-A4B-itsource62.4I 50C 81S 93K 50est.$0.047
  47. 66OpenAI DecisionsAPIClosed APIBase model: undisclosed62.4I 57C 90S 88K 49$0.052
  48. 67InstinctAPIClosed APIBase model: Qwen3.8-27Bsource*1162.2I 48C 93S 87K 64$0.015
  49. 68Blink v0.3 26B-A4BUsefine-tune · 26B-A4B · NVFP4Base model: google/gemma-4-26B-A4B-itsource61.1I 49C 91S 91K 50est.$0.048
  50. 69HS1 0.1.2APIClosed APIBase model: google/gemma-4-26B-A4B-it (self-reported)*1261.1I 53C 76S 94K 49est.$0.050
  51. 70baselabs elephant-1Usefine-tune · 9B · bf16Base model: Qwen/Qwen3.5-9Bsource60.9I 56C 85S 90K 48est.$0.052
  52. 71Diffusion JevUse26B / 4B activeBase model: google/diffusiongemma-26B-A4B-itsource59.3I 55C 61S 86K 50est.$0.048
  53. 72René-1 31B FP8Usefine-tune · 31B · FP8Base model: google/gemma-4-31B-itsource55.8I 62C 90S 87K 46est.$0.063
  54. 73Amazon Strands Decider 26B-A4BUseadapter · 26B-A4B · bf16Base model: google/gemma-4-26B-A4B-itsource54.5I 47C 87S 88K 50est.$0.045
  55. 74Wald 4B v2.1Usefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource54.3I 52C 87S 93K 46est.$0.062
  56. 75pplx-decider-flash-v1.2Usefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource54.0I 45C 78S 91K 64est.$0.015
  57. 76Bespoke Nimble 9B v3Useadapter · 9BBase model: Qwen/Qwen3.5-9Bsource53.5I 57C 77S 89K 46est.$0.063
  58. 77Aplomb 1Usefine-tune · 5.3BBase model: Qwen/Qwen3.5-4Bsource52.9I 45C 89S 89K 58est.$0.025
  59. 78Respan Span-02UseLLM decoderBase model: google/gemma-4-26B-A4B-itsource51.5I 61C 78S 86K 45est.$0.068
  60. 79Seb-9B-v2Usefine-tune · 9B · bf16Base model: Qwen/Qwen3.5-9Bsource50.0I 46C 84S 92K 49est.$0.051
  61. 80APUS-OpenJev-v1-9BUsefine-tune · 9BBase model: Qwen/Qwen3.5-9Bsource49.2I 47C 67S 82K 49est.$0.051
  62. 81Arcbright Elf 1 12BUsefine-tune · 12B · bf16Base model: google/gemma-4-12B-itsource48.8I 44C 86S 92K 57est.$0.026
  63. 82Jeffan-31b-1710Usefine-tune · 31B · bf16Base model: google/gemma-4-31B-itsource48.7I 72C 92S 85K 43est.$0.081
  64. 83Plumb-4BUsefine-tune · 4BBase model: alibiserikbay/JevK5source47.9I 43C 88S 94K 63est.$0.017
  65. 84SPX-CD ProAPISystemBase model: Qwen/Qwen3.8-27Bmodel card*1347.6I 65C 92S 77K 43est.$0.079
  66. 85Messier One v0.2Usefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource45.3I 42C 74S 93K 65$0.015
  67. 86spark-s1-4b-v6Useadapter · 4BBase model: Qwen3.5-4Bsource44.8I 43C 65S 86K 60est.$0.021
  68. 87KnowLine 31B-Gen1Usefine-tune · 31B · bf16Base model: google/gemma-4-31B-itsource44.4I 77C 93S 81K 41est.$0.093
  69. 88swanOneUseNVFP4Base model: Mia-AiLab/Qwen3.8-Flash-Next-NVFP4source43.8I 56C 89S 83K 42est.$0.085
  70. 89AJev lm6qUseadapter · 31B · bf16Base model: google/gemma-4-31B-itsource43.7I 71C 91S 82K 41est.$0.093
  71. 90Quyet-1.0-MediumUseadapter · 4BBase model: Qwen/Qwen3.5-4Bsource43.6I 42C 77S 89K 63est.$0.017
  72. 91Bespoke Nimble-v4APIClosed APIBase model: undisclosed43.2I 41C 87S 89K 61est.$0.020
  73. 92Turion Jev 31BUsefine-tune · 31B · fp8Base model: google/gemma-4-31B-itsource42.9I 71C 91S 86K 41est.$0.096
  74. 93Vansa-3.4APIClosed APIBase model: Qwen3.5-4B (self-reported)*1442.6I 41C 87S 95K 62$0.019
  75. 94JPT-4BUsemerge · 4BBase model: Qwen/Qwen3.5-4Bsource42.2I 41C 87S 88K 64est.$0.016
  76. 95levUseadapter · 4BBase model: Qwen/Qwen3.5-4Bsource42.1I 41C 82S 88K 62est.$0.019
  77. 96NInfer Qwen3.8-Flash-Next mixedUseLLM decoderBase model: Qwen3.8-Flash-Nextsource41.9I 49C 90S 89K 43est.$0.082
  78. 97decider-4b v2Useadapter · 4BBase model: Qwen/Qwen3.5-4B-Basesource41.1I 40C 90S 90K 65est.$0.015
  79. 98baselabs mammoth-1Usefine-tune · 27B · bf16Base model: Qwen/Qwen3.5-27Bsource41.1I 76C 94S 84K 40est.$0.104
  80. 99Jeffan-4B-2010Usefine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4Bsource41.1I 40C 88S 93K 64est.$0.016
  81. 100jev-localUse9BBase model: Qwen/Qwen3.5-9Bsource41.0I 42C 73S 73K 59est.$0.024
  82. 101GPT-6 Luna (low reasoning effort)APIClosed APIBase model: undisclosed40.4I 96C 98S 72K 39$0.108
  83. 102Autoloops – Gemma 4 31B ITAPIClosed APIBase model: google/gemma-4-31B-itvendor statement*1540.2I 58C 87S 84K 40$0.096
  84. 103Wald 4BUsefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource39.7I 41C 78S 82K 54est.$0.033
  85. 104GPT-6 Luna (default medium reasoning effort)APIClosed APIBase model: undisclosed39.1I 97C 100S 71K 38$0.113
  86. 105ClassOne Qwen 3.5 9BUse9BBase model: Qwen/Qwen3.5-9Bsource38.5I 42C 59S 90K 49est.$0.049
  87. 106JevK5 v0.3UseadapterBase model: Qwen/Qwen3.5-4Bsource37.4I 39C 90S 94K 63est.$0.017
  88. 107metask-jev-4bUse4BBase model: Qwen3.5-4Bsource37.3I 39C 84S 90K 58est.$0.026
  89. 108deck-4B v1.0Useadapter · FP8Base model: alibiserikbay/JevK5 (built on Qwen/Qwen3.5-4B)source 1source 237.3I 39C 89S 90K 63est.$0.017
  90. 109GLiClass v4 9BnewUsefine-tune · 9B · BF16Base model: Qwen/Qwen3.5-9Bsource37.0I 40C 87S 86K 51est.$0.042
  91. 110Kahn1 4BUsefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource37.0I 40C 84S 89K 50est.$0.046
  92. 111Clef-FlashUsefine-tuneBase model: Qwen/Qwen3.5-9BsourceWorkers AI price scenario (latency unmeasured): 39.1 (would be #104)36.6I 42C 85S 88K 47est.$0.059
  93. 112Decision-4BUseadapterBase model: Qwen/Qwen3.5-4Bsource35.4I 38C 80S 91K 66est.$0.014
  94. 113decisor-4bUsefine-tune · 4B · FP8Base model: Qwen/Qwen3.5-4Bsource35.3I 40C 52S 92K 64est.$0.016
  95. 114janus 4BUsefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource35.2I 38C 88S 93K 64est.$0.015
  96. 115Qwen3.5-9B Jev-like data-mix v2Useadapter · 9BBase model: Qwen/Qwen3.5-9Bsource33.6I 42C 80S 85K 46est.$0.065
  97. 116Seb-9BUsefine-tune · 9B · bf16Base model: Qwen/Qwen3.5-9Bsource32.6I 38C 88S 91K 49est.$0.051
  98. 117JevK5 v0.2.0Useadapter · 4BBase model: Qwen3.5-4Bsource32.1I 36C 89S 91K 63est.$0.017
  99. 118Tern-35B-A3BUsefine-tune · 35B-A3B · bf16Base model: Qwen/Qwen3.6-35B-A3Bsource32.1I 44C 89S 91K 42est.$0.085
  100. 119Amos 1 mediumAPI9BBase model: Qwen/Qwen3.5-9Bmodel card*1631.6I 38C 88S 86K 49est.$0.051
  101. 120Aleria Judge 1.1.1Use31B · bf16Base model: undisclosed31.2I 68C 86S 86K 36est.$0.138
  102. 121JevOneUseadapterBase model: Qwen/Qwen3.6-35B-A3Bsource31.1I 46C 92S 90K 40est.$0.101
  103. 122Fastino GLiDEAPIClosed APIBase model: undisclosed30.6I 66C 88S 77K 36$0.137
  104. 123Jebadiah 9B v2.1Usemerge · 9BBase model: Qwen/Qwen3.5-9Bsource30.3I 37C 90S 91K 49est.$0.051
  105. 124Verdin-4BUsemerge · 4BBase model: Qwen/Qwen3.5-4Bsource29.9I 35C 84S 90K 62est.$0.018
  106. 125TypeCastLM 1.4.0Use3.8BBase model: Qwen/Qwen3.5-4Bsource29.8I 35C 77S 93K 65est.$0.015
  107. 126blink-4bUsemerge · 4BBase model: Qwen/Qwen3.5-4Bsource29.5I 35C 88S 91K 64est.$0.016
  108. 127jiwo 4bUsefine-tune · 4.2BBase model: Qwen/Qwen3.5-4Bsource28.9I 35C 86S 92K 64est.$0.016
  109. 128Imajev-4BUseadapter · 4BBase model: Qwen/Qwen3.5-4Bsource28.6I 35C 89S 90K 63est.$0.017
  110. 129SimpleJev Qwen3.6-35B-A3BAPI35B-A3BBase model: Qwen/Qwen3.6-35B-A3Bvendor statement*1727.5I 56C 80S 76K 35est.$0.145
  111. 130Uniphore-4B-DeciderUsefine-tune · 4BBase model: undisclosed27.2I 34C 80S 87K 65est.$0.015
  112. 131Decision 2.0 Lux 9BUseLLM decoderBase model: vllm-sr/Decision-1.0-Lux-9Bsource27.1I 36C 87S 90K 48est.$0.052
  113. 132HopperUseadapter · 4BBase model: Qwen/Qwen3.5-4Bsource26.5I 34C 89S 91K 62est.$0.018
  114. 133Malkuth-4BUseadapter · 4BBase model: Qwen/Qwen3.5-4B-Basesource26.2I 34C 88S 89K 56est.$0.030
  115. 134jqvUse32BBase model: Qwen3-32Bsource25.4I 34C 90S 82K 51est.$0.042
  116. 135JPT-35B-A3BUsemerge · 35B-A3BBase model: Qwen/Qwen3.5-35B-A3Bsource25.4I 51C 90S 90K 34est.$0.163
  117. 136decider-35b-a3bUsefine-tune · 35B-A3BBase model: Qwen/Qwen3.5-35B-A3B-Basesource24.8I 49C 84S 91K 34est.$0.154
  118. 137CoCo-Decision-4BUsefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource24.6I 33C 81S 91K 65est.$0.014
  119. 138Manchego v2.1Useadapter · 4BBase model: Qwen/Qwen3.5-4Bsource24.5I 33C 84S 91K 64est.$0.015
  120. 139Decision 4B v1.2Useadapter · 4BBase model: Qwen/Qwen3.5-4Bsource24.5I 32C 88S 94K 63est.$0.017
  121. 140Standard One 3B SH v4Usefine-tune · 3B · bf16Base model: StandardThinking/StandardOne-3Bsource24.4I 34C 78S 93K 49est.$0.052
  122. 141Fastino GLiDE 1.2Usefine-tune · 27B · bf16Base model: undisclosed22.0I 76C 93S 85K 31est.$0.205
  123. 142Raw Qwen3 4B Instruct 2507 direct logitsUseoriginal · 4BBase model: Qwen/Qwen3-4B-Instruct-2507 (unmodified)source21.8I 34C 35S 91K 63est.$0.017
  124. 143StartLux-Decision v1.1 27B InstantUsefine-tune · 27BBase model: undisclosed21.6I 73C 92S 90K 30est.$0.209
  125. 144Standard One 3B SHUsefine-tune · 3BBase model: StandardThinking/StandardOne-3Bsource21.6I 33C 78S 86K 49est.$0.052
  126. 145RSI-Jev v6.1-VL 27BUsemerge · 27BBase model: Qwen/Qwen3.8-27Bsource21.6I 62C 85S 85K 31est.$0.197
  127. 146GPT-5.6 LunaAPIClosed APIBase model: undisclosed21.4I 93C 94S 73K 30$0.213
  128. 147JEV-27BUsedistilled · 27BBase model: Qwen/Qwen3.8-27Bsource21.3I 57C 91S 86K 31est.$0.199
  129. 148torchcast-decision-27bUsemerge · 27BBase model: startlux-models/StartLux-Decision-27Bsource 1source 2*1821.1I 70C 90S 88K 30est.$0.210
  130. 149Bespoke Nimble 9BUseadapter · 9BBase model: Qwen3.5-9Bsource21.1I 43C 70S 86K 37est.$0.128
  131. 150Nimble-v4-maxUsefine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource20.7I 71C 91S 85K 30est.$0.214
  132. 151pplx-decider-v1.2Usefine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource20.6I 72C 90S 86K 30est.$0.215
  133. 152Perplexity Decider v1.1 27BUsefine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource20.6I 76C 90S 86K 30est.$0.217
  134. 153vLLM-SR d3-turboUsefine-tune · 27BBase model: undisclosed20.5I 75C 90S 86K 30est.$0.218
  135. 154vLLM-SR d3Usefine-tune · 26BBase model: Qwen/Qwen3.8-27Bsource20.5I 73C 91S 87K 30est.$0.218
  136. 155reflex 4BUseadapter · 4BBase model: Qwen/Qwen3.5-4Bsource20.4I 31C 87S 69K 63est.$0.017
  137. 156Vera LargeUsefine-tune · 27B · bf16Base model: Qwen/Qwen3.8-27Bsource20.3I 68C 91S 88K 30est.$0.217
  138. 157Amos 1 largeAPI27BBase model: Qwen/Qwen3.8-27Bauthor self-reportmodel card*1920.3I 68C 88S 83K 30est.$0.215
  139. 158JADEUseadapter · 27BBase model: Qwen/Qwen3.8-27Bsource20.2I 48C 51S 85K 34est.$0.164
  140. 159Nagi-ENORMOUS 27BUseadapter · 27BBase model: Qwen/Qwen3.8-27Bsource19.8I 62C 85S 86K 30est.$0.216
  141. 160typecastlmUse4BBase model: Qwen3.5-4Bsource19.7I 30C 83S 93K 64est.$0.016
  142. 161Jebadiah 27BUsemerge · 27BBase model: Qwen/Qwen3.8-27Bsource19.2I 57C 83S 85K 30est.$0.217
  143. 162iota-decide-4bUsefine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4B-Basesource18.9I 29C 85S 87K 63est.$0.017
  144. 163Decision 2.0 Vega 27BUseadapter · 27BBase model: Qwen/Qwen3.8-27Bsource 1source 218.9I 60C 82S 83K 30est.$0.222
  145. 164Jebadiah 9B v2Usemerge · 9BBase model: Qwen/Qwen3.5-9Bsource18.8I 31C 83S 91K 49est.$0.051
  146. 165Standard One 8B SHUsefine-tune · 8BBase model: StandardThinking/StandardOne-8Bsource18.7I 34C 85S 84K 43est.$0.078
  147. 166Decision 4B v1.1Useadapter · 4BBase model: Qwen/Qwen3.5-4Bsource18.7I 29C 85S 94K 63est.$0.017
  148. 167AutoJev-27BUsefine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource18.5I 56C 88S 88K 29est.$0.226
  149. 168pjev-2bUsefine-tune · 2.5BBase model: openbmb/MiniCPM5-2Bsource18.1I 29C 84S 93K 64est.$0.016
  150. 169Eikos-27BUseadapter · 27BBase model: Qwen/Qwen3.8-27Bsource18.1I 64C 90S 88K 29est.$0.238
  151. 170Instinct Dual 4BAPIClosed APIBase model: Qwen3.5-4Bsource*2018.0I 28C 86S 89K 75$0.0068
  152. 171NInfer Qwen3.8-27B NVFP4Use27BBase model: Qwen3.8-27Bsource17.7I 52C 84S 90K 29est.$0.231
  153. 172Compass 0.2.0Useadapter · 4BBase model: Qwen/Qwen3.5-4Bsource17.6I 29C 83S 83K 58est.$0.026
  154. 173JEB-35B-A3BUsefine-tune · 35B-A3BBase model: Qwen/Qwen3.6-35B-A3Bsource17.3I 33C 90S 93K 43est.$0.080
  155. 174OpenJevUse26B-A4BBase model: google/diffusiongemma-26B-A4B-itsource 1source 217.3I 70C 81S 73K 29est.$0.241
  156. 175Open-Jev 9BUseadapter · 9BBase model: Qwen/Qwen3.5-9Bsource17.0I 44C 80S 74K 33est.$0.170
  157. 176Gemini 3.1 Flash-LiteAPIClosed APIBase model: Gemini 3 Pro (Google)model card*2116.9I 59C 61S 79K 29$0.230
  158. 177Standard One 8BUseadapter · 8BBase model: mistralai/Ministral-3-8B-Instruct-2512-BF16source16.9I 33C 84S 92K 43est.$0.078
  159. 178ClefUsefine-tuneBase model: Qwen/Qwen3.8-27BsourceWorkers AI price scenario (latency unmeasured): 29.4 (would be #127)16.7I 59C 92S 83K 28est.$0.249
  160. 179ArseneLupin v1.1Useadapter · 4BBase model: Qwen/Qwen3.5-4Bsource16.5I 28C 82S 90K 61est.$0.020
  161. 180system-one-openAPIadapter · 2BBase model: Gemma 4 E2Bsource16.4I 28C 76S 79K 68est.$0.011
  162. 181JEV Qwen3.5-9B Base NVFP4Use9B · NVFP4Base model: ig1/Qwen3.5-9B-NVFP4source16.0I 29C 85S 93K 48est.$0.056
  163. 182Arbiter-4BUsefine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4Bsource15.9I 27C 77S 92K 66est.$0.013
  164. 183Decider-4BUsefine-tune · 4B · bf16Base model: Qwen/Qwen3.5-4Bsource15.7I 27C 87S 92K 66est.$0.013
  165. 184ClassOne Gemma 4 E4BUsefine-tune · E4BBase model: google/gemma-4-E4B-itsource15.6I 28C 49S 92K 69est.$0.011
  166. 185SemIfUse4BBase model: Qwen/Qwen3.5-4Bsource15.5I 27C 84S 90K 63est.$0.017
  167. 186openjev-sglangAPI35B-A3BBase model: Qwen/Qwen3.6-35B-A3Bsource15.4I 39C 79S 78K 36est.$0.140
  168. 187DecisionNode Edge 0.9.2Usefine-tune · 2B · bf16Base model: Qwen/Qwen3.5-2B-Basesource15.3I 27C 77S 93K 71est.$0.0091
  169. 188RYOTIDE-Qwen9Usepipeline · 9B · BF16Base model: Qwen/Qwen3.5-9Bsource14.7I 28C 77S 86K 49est.$0.049
  170. 189ClassOne Qwen 3.5 4BUsefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource14.3I 27C 49S 91K 65est.$0.015
  171. 190Jez-large-1.0.0Usefine-tune · E4B · bf16Base model: google/gemma-4-E4B-itsource13.8I 26C 83S 91K 71est.$0.0096
  172. 191RYOTIDE-QwenUsepipeline · 4BBase model: undisclosed12.9I 25C 87S 89K 62est.$0.018
  173. 192Chon-1.0-12BUsefine-tune · 12BBase model: google/gemma-4-12B-itsource12.8I 27C 59S 65K 49est.$0.050
  174. 193NomaUseLLM decoderBase model: Qwen/Qwen3.5-4B-Basesource12.7I 25C 84S 93K 62est.$0.019
  175. 194KnowLine 35B-A3B-Gen1.5Usefine-tune · 35B-A3B · fp8Base model: Qwen/Qwen3.5-35B-A3Bsource12.4I 60C 93S 92K 25est.$0.323
  176. 195Bev / Bonsai 27BUse27B · GGUF PQ2_0Base model: Qwen/Qwen3.8-27Bsource11.7I 44C 89S 73K 28est.$0.247
  177. 196LitJevUse27BBase model: Qwen/Qwen3.8-27Bsource11.5I 43C 90S 68K 28est.$0.244
  178. 197local-jev Qwen3.5-4BUsefine-tune · 4BBase model: Qwen/Qwen3.5-4Bsource11.4I 24C 86S 86K 59est.$0.023
  179. 198system-oneUse8BBase model: Qwen3-8Bsource11.0I 29C 30S 91K 45est.$0.068
  180. 199StartLux-Decision v1.1 27B AutoThinkUsefine-tune · 27BBase model: undisclosed10.8I 67C 91S 75K 24est.$0.354
  181. 200kev 8BUseadapter · 8BBase model: Qwen/Qwen3-8B-Basesource10.6I 29C 71S 88K 40est.$0.097
  182. 201Decision 2BUseadapter · 2BBase model: openbmb/MiniCPM5-2Bsource10.3I 23C 87S 90K 66est.$0.013
  183. 202OpenSourceJevUseadapter · 4B · GGUF Q4_K_MBase model: Qwen3.5-4Bsource10.2I 23C 74S 76K 69est.$0.011
  184. 203Imajev-9BUseadapter · 9BBase model: Qwen/Qwen3.5-9Bsource9.8I 37C 88S 79K 31est.$0.203
  185. 204open-alternative-jevUse4BBase model: Qwen3.5-4Bsource9.4I 22C 76S 91K 63est.$0.017
  186. 205Raw Qwen3 8B direct logitsUseoriginal · 8BBase model: Qwen/Qwen3-8B-Basesource9.3I 27C 34S 88K 46est.$0.065
  187. 206Liquid AI d1-3BUsefine-tune · 3BBase model: LiquidAI/LFM2.5-VL-3Bsource8.9I 22C 77S 94K 51est.$0.044
  188. 207Krino-2BUseadapter · 2BBase model: Qwen/Qwen3.5-2B-Basesource8.7I 22C 63S 91K 61est.$0.020
  189. 208classify-1APIClosed APIBase model: undisclosed8.5I 22C 83S 84K 51$0.044
  190. 209Imajev-2BUseadapter · 2BBase model: Qwen/Qwen3.5-2Bsource7.9I 21C 87S 81K 52est.$0.041
  191. 210decider-2bUseadapter · 2BBase model: Qwen/Qwen3.5-2B-Basesource7.7I 21C 69S 94K 65est.$0.015
  192. 211Nemotron Diffusion 8BUse8BBase model: nvidia/Nemotron-Labs-Diffusion-8Bsource7.3I 20C 75S 94K 55est.$0.030
  193. 212D1A on Gemma 4 E4BUsefine-tune · BF16Base model: google/gemma-4-E4Bsource7.1I 20C 82S 92K 61est.$0.021
  194. Nev-2B (re-measure pending under the 10 s rule)Usedistilled · 2B · GGUF Q8_0Base model: Qwen/Qwen3.5-2B-Basesource6.9I 20C 80S 55K 71est.$0.0092
  195. 213DeepSeek V4.1 FlashAPILLM decoderBase model: undisclosed6.9I 84C 95S 66K 20$0.480
  196. 214kev 4BUseadapter · 4BBase model: Qwen/Qwen3-4B-Basesource*226.6I 20C 68S 90K 66est.$0.014
  197. 215Clef-omniUsefine-tune · 30B-A3BBase model: Qwen/Qwen3-Omni-30B-A3B-Instructsource6.1I 34C 82S 87K 27est.$0.276
  198. 216Kev 27BUsefine-tune · 27BBase model: Qwen/Qwen3.8-27Bsource5.8I 64C 94S 83K 18est.$0.529
  199. 217FrontiersMind Lumma-fev-27BUsefine-tune · 27B · bf16Base model: Qwen/Qwen3.8-27Bsource5.3I 55C 89S 85K 18est.$0.544
  200. 218DecisionNode Edge Mini 0.9.2Usefine-tune · 0.8B · bf16Base model: Qwen/Qwen3.5-0.8B-Basesource5.3I 18C 78S 93K 80est.$0.0046
  201. 219jiwo 0.8bUsefine-tune · 0.75BBase model: Qwen/Qwen3.5-0.8Bsource4.1I 16C 79S 94K 78est.$0.0052
  202. 220Malkuth-2BUseadapter · 2BBase model: empero-ai/Qwen3.8-2B-Distillsource4.0I 16C 79S 92K 66est.$0.014
  203. 221Qwen3-Reranker-4BUseRerankerBase model: Qwen/Qwen3-4B-Basesource3.7I 16C 73S 81K 48est.$0.052
  204. 222SimpleJev Qwen3.8-27B (self-hosted, v1.6 pool)Use27BBase model: Qwen/Qwen3.8-27Bsource3.4I 58C 92S 76K 15est.$0.676
  205. 223ZeroEntropy zerank-2UseRerankerBase model: Qwen/Qwen3-4Bsource3.3I 16C 83S 82K 48est.$0.052
  206. 224EXAONE-4.0-1.2B-JEV v0.3Use1.2BBase model: LGAI-EXAONE/EXAONE-4.0-1.2Bsource3.3I 15C 81S 94K 58est.$0.025
  207. 225Open-Jev 2BUseadapter · 2BBase model: Qwen/Qwen3.5-2Bsource3.2I 22C 74S 77K 33est.$0.170
  208. 226SimpleJev Qwen3.8-27BAPI27BBase model: Qwen/Qwen3.8-27Bsource3.2I 53C 90S 76K 15est.$0.687
  209. 227Vega 4BUseadapter · 4BBase model: undisclosed3.1I 16C 57S 84K 47est.$0.058
  210. 228Open-Jev 27B v1.1Useadapter · 27BBase model: Qwen/Qwen3.8-27Bsource2.9I 52C 83S 72K 14est.$0.716
  211. 229Raw Phi-4 mini direct logitsUseoriginalBase model: microsoft/Phi-4-mini-instruct (unmodified)source2.5I 14C 57S 92K 53est.$0.036
  212. 230decision-machine-1APIClosed APIBase model: Qwen2.5-1.5B-Instruct (choices) + GLiNER2.5 (spans)vendor statement*232.4I 13C 79S 93K 58$0.024
  213. 231SimpleJevUse0.8BBase model: Qwen/Qwen3.5-0.8Bsource2.0I 13C 49S 59K 70est.$0.0098
  214. 232Jeetu Decider 2BUsefine-tune · 2B · bf16Base model: Qwen/Qwen3.5-2B-Basesource1.2I 10C 73S 91K 70est.$0.0098
  215. 233JevActAPIClosed APIBase model: Qwen3.5-2Bauthor self-report*241.1I 10C 47S 76K 69est.$0.011
  216. 234D1A on Gemma 4 E2BUsefine-tune · BF16Base model: google/gemma-4-E2Bsource0.8I 9C 82S 93K 61est.$0.021
  217. 235smalljev semantic-v9Useadapter · 2BBase model: openbmb/MiniCPM5-2B-Basesource0.8I 9C 77S 90K 61est.$0.020
  218. 236Gutsy 0.8B v0.3Usefine-tune · 0.8B · GGUF Q8_0Base model: Qwen/Qwen3.5-0.8Bsource0.8I 9C 76S 72K 78est.$0.0053
  219. 237Vega-U (Qwen3.5-4B reader)Useadapter · 4BBase model: undisclosed0.7I 8C 77S 89K 65est.$0.015
  220. 238PurpleMIST-Mini-1.0Usefine-tune · 2BBase model: Qwen/Qwen3.5-2B-Basesource0.7I 8C 78S 89K 71est.$0.0094
  221. 239kev 0.6BUseadapter · 0.6BBase model: Qwen/Qwen3-0.6B-Basesource0.5I 8C 66S 91K 80est.$0.0046
  222. 240jul fastUse2BBase model: openbmb/MiniCPM5-2Bsource0.5I 7C 78S 91K 69est.$0.011
  223. 241OpenDecisionUseEncoder / classifierBase model: MoritzLaurer/ModernBERT-large-zeroshot-v2.0source0.5I 7C 69S 86K 79est.$0.0050
  224. 242ClassOne Gemma 4 E2BUseE2BBase model: google/gemma-4-E2B-itsource0.5I 8C 23S 93K 70est.$0.0098
  225. Qwen3.5-0.8B Decision Model (re-measure pending under the 10 s rule)Usefine-tune · 0.8BBase model: Qwen/Qwen3.5-0.8B-Basesource0.5I 7C 78S 56K 80est.$0.0048
  226. 243Fastino GLiNER-2.5-DecideAPIfine-tune · 340MBase model: fastino/gliner2-large-v1source 1source 2*250.3I 7C 62S 85K 46$0.064
  227. 244Deem 0.8B v1UseLLM decoderBase model: Qwen/Qwen3.5-0.8Bsource0.3I 7C 30S 85K 80est.$0.0045
  228. 245Decision FastUseadapter · 0.6BBase model: Qwen/Qwen3-0.6B-Basesource0.3I 6C 75S 91K 80est.$0.0046
  229. 246Quyet-1.0-Small-ENUsefine-tune · 153MBase model: answerdotai/ModernBERT-basesource0.3I 6C 73S 95K 89est.$0.0023
  230. 247ClassOne Qwen 3.5 2BUsefine-tune · 2BBase model: Qwen/Qwen3.5-2Bsource0.2I 6C 33S 93K 70est.$0.0098
  231. 248Laya typed-decisionsUsefine-tune · 421MBase model: ModernBERT-largesource0.2I 5C 83S 69K 83est.$0.0038
  232. 249Vega-U (Qwen3.5-0.8B reader)Useadapter · 0.8BBase model: undisclosed0.2I 5C 67S 73K 79est.$0.0049
  233. 250WaterSheepUseEncoder / classifierBase model: answerdotai/ModernBERT-basesource0.2I 5C 66S 77K 69est.$0.011
  234. 251openJev Verdict 1.4UseEncoder / classifierBase model: knowledgator/gliclass-modern-base-v2.0source0.2I 5C 76S 81K 87est.$0.0028
  235. 252Raw Qwen3 0.6B direct logitsUseoriginal · 0.6BBase model: Qwen/Qwen3-0.6B-Basesource0.2I 6C 11S 93K 78est.$0.0056
  236. 253lev-350mUseadapterBase model: LiquidAI/LFM2.5-350Msource0.2I 5C 82S 94K 80est.$0.0046
  237. 254Raw Qwen3 1.7B direct logitsUseoriginal · 1.7BBase model: Qwen/Qwen3-1.7B-Basesource0.1I 5C 12S 93K 69est.$0.011
  238. 255kev 0.5BUseadapter · 0.5BBase model: Qwen/Qwen2.5-0.5Bsource0.1I 5C 61S 92K 80est.$0.0046
  239. 256Jez-small-1.0.0Usefine-tune · 0.8B · fp32Base model: Qwen/Qwen3.5-0.8Bsource0.1I 5C 82S 93K 80est.$0.0048
  240. 257openJev VerdictUsefine-tuneBase model: knowledgator/gliclass-modern-base-v2.0source 1source 20.1I 4C 49S 78K 87est.$0.0028
  241. 258Quyet-1.0-SmallUsefine-tune · 328MBase model: aisingapore/SEA-LION-ModernBERT-300Msource0.1I 4C 61S 95K 80est.$0.0048
  242. Reflex Instinct 0.6B (re-measure pending under the 10 s rule)Useadapter · 0.6BBase model: Qwen/Qwen3-0.6Bsource0.1I 4C 73S 59K 80est.$0.0047
  243. 259watt-flash-0.1Usefine-tune · 140MBase model: jhu-clsp/mmBERT-smallsource0.1I 4C 79S 74K 89$0.0023
  244. 260Quyet-1.0-TinyUsedistilled · 183MBase model: jhu-clsp/mmBERT-smallsource0.1I 4C 58S 95K 88est.$0.0024
  245. 261Phocinae-Largha-150M-v1Usefine-tune · 144MBase model: undisclosed0.1I 4C 57S 77K 95est.$0.0014
  246. 262Liquid AI d1-omni-600MUsefine-tune · 600MBase model: LiquidAI/LFM2.5-Encoder-350Msource0.0I 3C 62S 94K 73est.$0.0079
  247. 263Vega 0.8BUseadapter · 0.8BBase model: Qwen/Qwen3.5-0.8Bsource0.0I 3C 63S 89K 61est.$0.019
  248. 264open-jev-deberta-v3-largeUseadapterBase model: microsoft/deberta-v3-largesource0.0I 3C 68S 67K 78est.$0.0056
  249. modernbert-jev v2 (re-measure pending under the 10 s rule)Usefine-tune · 150M · ONNX INT8Base model: undisclosed0.0I 3C 67S 58K 70est.$0.0097
  250. 265Julia-1Usefine-tune · 144MBase model: jhu-clsp/mmBERT-smallsource0.0I 2C 17S 78K 90est.$0.0022
  251. 266verdict-smallUsefine-tune · 118MBase model: intfloat/multilingual-e5-smallsource0.0I 2C 73S 89K 100est.$0.0009
  252. 267Tacet SonataUsefine-tune · 144MBase model: jhu-clsp/mmBERT-smallsource0.0I 2C 46S 79K 89est.$0.0023
  253. 268LayaUsefine-tune · 421MBase model: ModernBERT-largesource0.0I 2C 63S 73K 85est.$0.0032
  254. 269Mixedbread mxbai-rerank-base-v2UseRerankerBase model: Qwen2.5 (size not stated)source 1source 20.0I 1C 87S 92K 60est.$0.021
  255. 270Laya multilingualUsefine-tune · 322MBase model: mmBERT-basesource0.0I 0C 34S 77K 82est.$0.0039
  256. 271Certo v1UseEncoder / classifierBase model: ModernBERT-largesource0.0I 0C 86S 92K 97est.$0.0013
  257. 272CLM-8BUsefine-tune · 8BBase model: Qwen/Qwen3-8Bsource0.0I 0C 40S 92K 51est.$0.045
  258. 273BAAI bge-reranker-v2-m3Usefine-tuneBase model: BAAI/bge-m3source0.0I 0C 88S 94K 59est.$0.023
  259. 274Alibaba GTE Reranker ModernBERT-baseUsefine-tuneBase model: answerdotai/ModernBERT-basesource0.0I 0C 83S 94K 69est.$0.011
  260. 275MirrorUseEncoder / classifierBase model: undisclosed0.0I 0C 52S 74K 89est.$0.0023
  261. 276Open Jev JSON CanvasUse26B-A4BBase model: google/diffusiongemma-26B-A4B-itsource0.0I 61C 0S 85K 49est.$0.049
  262. 277Qwen3.8 27BAPI27BBase model: Qwen/Qwen3.8-27Bmodel card*260.0I 33C 64S 55K 0est.$2.178

Base-model notes

  1. *1 Super Decider (base model: google/diffusiongemma-26B-A4B-it): Vendor model card; the hosted endpoint was not inspected.
  2. *2 Myles-G 12B (base model: google/gemma-4-12B-it): Public model-card metadata; weights gated, deployed endpoint not inspected.
  3. *3 Fastino GLiDE (base model: undisclosed): No public base model declared; proprietary vendor-hosted API.
  4. *4 Drex Max (base model: Qwen3.8-27B): Vendor's technical report. The vendor updated the model behind this API id after our 21 Oct measurement.
  5. *5 Bespoke Nimble-v4-max (base model: Qwen/Qwen3.8-27B): Vendor model card (published 18 Oct, after our 14 Oct measurement of the hosted API).
  6. *6 Super Decider (base model: google/diffusiongemma-26B-A4B-it): Vendor model card; the hosted endpoint was not inspected.
  7. *7 Myles-G 31B (base model: google/gemma-4-31B-it): Public model-card metadata; weights gated, deployed endpoint not inspected.
  8. *8 Microsoft-Decision-1 (base model: Qwen3.5-9B): Base model disclosed by Microsoft; the Decision-1 endpoint is evaluated as a hosted API offering.
  9. *9 SPX-CD Flash (base model: Qwen/Qwen3.6-35B-A3B): Vendor model card for the published adapter (9 Oct); the hosted Oct-4 checkpoint measured here is not stated to be identical.
  10. *10 Surogate Rune 26B-A4B v3 (base model: google/gemma-4-26B-A4B-it): This model also powers our System1 Models s1-pro service.
  11. *11 Instinct (base model: Qwen3.8-27B): Operator-reported; weights not publicly verifiable.
  12. *12 HS1 0.1.2 (base model: google/gemma-4-26B-A4B-it (self-reported)): Customer statement (order notes, 21 Oct 2026): a LoRA fine-tune of Gemma-4-26B-A4B-it served in FP8. Not publicly documented and not independently verified.
  13. *13 SPX-CD Pro (base model: Qwen/Qwen3.8-27B): Vendor model card for the published adapter (9 Oct); the hosted Oct-4 checkpoint measured here is not stated to be identical.
  14. *14 Vansa-3.4 (base model: Qwen3.5-4B (self-reported)): Developer-reported to Benchmark Heaven (private correspondence, 1 Oct 2026): built on Qwen3.5-4B via an open decision fine-tune, with Vansa's own adapters; the intermediate fine-tune is not named. Not publicly documented or independently verified.
  15. *15 Autoloops – Gemma 4 31B IT (base model: google/gemma-4-31B-it): Vendor page.
  16. *16 Amos 1 medium (base model: Qwen/Qwen3.5-9B): Model card; the hosted endpoint was not inspected.
  17. *17 SimpleJev Qwen3.6-35B-A3B (base model: Qwen/Qwen3.6-35B-A3B): Vendor README.
  18. *18 torchcast-decision-27b (base model: startlux-models/StartLux-Decision-27B): Direct base is StartLux-Decision-27B (startlux-models, CC-BY-NC-4.0), itself a modified Alibaba Qwen model; its card does not name the Qwen version. Qwen3.8-27B is inferred from the identical text_config.
  19. *19 Amos 1 large (base model: Qwen/Qwen3.8-27B): Vendor statement and model card; the hosted endpoint was not inspected.
  20. *20 Instinct Dual 4B (base model: Qwen3.5-4B): Operator-reported; weights not publicly verifiable.
  21. *21 Gemini 3.1 Flash-Lite (base model: Gemini 3 Pro (Google)): Google's own model.
  22. *22 kev 4B (base model: Qwen/Qwen3-4B-Base): Evaluated Qwen3 variant; later releases use a different base.
  23. *23 decision-machine-1 (base model: Qwen2.5-1.5B-Instruct (choices) + GLiNER2.5 (spans)): Vendor blog (19 Sep 2026); the vendor says the backend can change.
  24. *24 JevAct (base model: Qwen3.5-2B): Author's issue title only; not otherwise documented.
  25. *25 Fastino GLiNER-2.5-Decide (base model: fastino/gliner2-large-v1): Measured via Fastino's hosted API. A 340M English classifier (8k context), not built for multi-step reasoning: near chance on Noul and Score items; 23 very long items exceed its context. Our request follows Fastino's documented format. The open checkpoint scores 11.2 on the community Decision Index (rank 53 of 70; Jev 57.9). Fastino's larger GLiDE is a separate model, ranked separately on the API board.
  26. *26 Qwen3.8 27B (base model: Qwen/Qwen3.8-27B): The open model itself, not a fine-tune.
Weights:
Adjust weights ↓

Weights are relative: each axis counts in proportion to its slider. The score stays a weighted harmonic mean with the low-axis gates; the gates still apply when an axis sits at 0. Only equal weights give the official JevBench Score and rank.

Score = 4 / (1/I + 1/C + 1/S + 1/K) (each 0–100; × (axis / 50)² for Intelligence, Speed or Cost below 50)

  • Jev — reference (TypeSafe, closed) (2)
  • Closed API (weights not public) (32)
  • Open weights · LLM decoder (208)
  • Open weights · diffusion LM (6)
  • Open weights · encoder / classifier (21)
  • Open weights · reranker (5)
  • Base model control (no decision fine-tune, raw logits) (5)
  • System (router / cascade / ensemble) (3)
  • Striped bar = same system under the labelled alternative price assumption
  • Shown, not ranked
I, C, S, K = Intelligence, Calibration, Speed, Cost; the est. pill = estimated cost; ann. = announced price; API = the operator's endpoint saw held-out benchmark inputs, without answers; $/1k decisions = US dollars per 1,000 decisions (not heat-shaded). Names link to each project.

Compare two systems

Pick any two. Radars for the score axes, capability by subject topic and use cases, chance-corrected competence per request type on the open and sealed sets, and competence per tier on each set. Further out is better on every spoke; the link keeps the pair.

  • A: Jev 1.13.0REFERENCE · TypeSafe — Jev — reference (TypeSafe, closed) · Score 71.4 (#23)
  • B: Super Decider — Open weights · diffusion LM · Score 78.5 (#1)

The four score axes

Radar: the four score axes, two systemsThe four score axes, Jev 1.13.0 vs Super Decider. Intelligence: 63.6 vs 78.9; Calibration: 90.6 vs 90.8; Speed: 91.2 vs 85.3; Cost: 54.7 vs 64.3.50100Intelligence: Jev 1.13.0: 63.6; Super Decider: 78.9Intelligence63.6 · 78.9Calibration: Jev 1.13.0: 90.6; Super Decider: 90.8Calibration90.6 · 90.8Speed: Jev 1.13.0: 91.2; Super Decider: 85.3Speed91.2 · 85.3Cost: Jev 1.13.0: 54.7; Super Decider: 64.3Cost54.7 · 64.3
0–100, the values in the table. An axis a system has no published value for is left as a gap (it counts as 0 in the composite).

Capability by subject topic

Super Decider: No per-category values are published for this system.

Radar: capability by subject topic, two systemsCapability by subject topic, Jev 1.13.0 vs Super Decider. Math & numbers: 40.5 vs No per-category values are published for this system.; Coding & software: 80.3 vs No per-category values are published for this system.; Rules, policy & law: 73.7 vs No per-category values are published for this system.; Finance & commerce: 83.4 vs No per-category values are published for this system.; Support & operations: 91.5 vs No per-category values are published for this system.; Everyday language: 96.1 vs No per-category values are published for this system.; Safety & security: 67.7 vs No per-category values are published for this system.. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Math & numbers: Jev 1.13.0: 40.5; Super Decider: No measured result — Math & numbers: a calculation decides the answer: arithmetic, word problems, probability, dates, units. 235 items (174 open, 61 sealed).Math &numbers40.5 · n/aCoding & software: Jev 1.13.0: 80.3; Super Decider: No measured result — Coding & software: code, SQL, repositories, developer tools and IT systems. 92 items (67 open, 25 sealed).Coding &software80.3 · n/aRules, policy & law: Jev 1.13.0: 73.7; Super Decider: No measured result — Rules, policy & law: applying written rules: company policies, contracts, regulations, eligibility. 548 items (260 open, 288 sealed).Rules, policy& law73.7 · n/aFinance & commerce: Jev 1.13.0: 83.4; Super Decider: No measured result — Finance & commerce: money: payments, refunds, invoices, orders, expenses, insurance payouts. 224 items (134 open, 90 sealed).Finance &commerce83.4 · n/aSupport & operations: Jev 1.13.0: 91.5; Super Decider: No measured result — Support & operations: support tickets, incidents, logistics, scheduling desks and routing work to a team. 282 items (131 open, 151 sealed).Support &operations91.5 · n/aEveryday language: Jev 1.13.0: 96.1; Super Decider: No measured result — Everyday language: short everyday messages: intents, assistant requests, reading a detail out of a text. 115 items (98 open, 17 sealed).Everydaylanguage96.1 · n/aSafety & security: Jev 1.13.0: 67.7; Super Decider: No measured result — Safety & security: untrusted or injected instructions, fraud, moderation, access and security triage. 128 items (40 open, 88 sealed).Safety &security67.7 · n/a

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

What the decision is about — math and dates, coding, rules, money, support work, everyday messages, safety. Chance-corrected competence per category (0 = chance, below chance negative, 100 = perfect), Jev 1.13.0: open+sealed; Super Decider: open+sealed items pooled; only categories with at least 30 items are spokes, smaller ones are listed below. Hover a category for its definition and item count.

No per-category values are published for this pair.

What each category means · items per category
  • Math & numbers — a calculation decides the answer: arithmetic, word problems, probability, dates, units. 235 items (174 open / 61 sealed)
  • Coding & software — code, SQL, repositories, developer tools and IT systems. 92 items (67 open / 25 sealed)
  • Rules, policy & law — applying written rules: company policies, contracts, regulations, eligibility. 548 items (260 open / 288 sealed)
  • Finance & commerce — money: payments, refunds, invoices, orders, expenses, insurance payouts. 224 items (134 open / 90 sealed)
  • Support & operations — support tickets, incidents, logistics, scheduling desks and routing work to a team. 282 items (131 open / 151 sealed)
  • Everyday language — short everyday messages: intents, assistant requests, reading a detail out of a text. 115 items (98 open / 17 sealed)
  • Safety & security — untrusted or injected instructions, fraud, moderation, access and security triage. 128 items (40 open / 88 sealed)

Use cases (TypeSafe categories)

Super Decider: No per-category values are published for this system.

Radar: use cases (typesafe categories), two systemsUse cases (TypeSafe categories), Jev 1.13.0 vs Super Decider. Search & retrieval: 95.9 vs No per-category values are published for this system.; Model routing: 96.5 vs No per-category values are published for this system.; LLM guardrails: 67.0 vs No per-category values are published for this system.; Lead generation: 88.1 vs No per-category values are published for this system.; Customer support: 91.9 vs No per-category values are published for this system.; Insurance claims: 67.5 vs No per-category values are published for this system.; Financial crime: 58.3 vs No per-category values are published for this system.; Legal & compliance: 74.0 vs No per-category values are published for this system.; E-commerce: 89.8 vs No per-category values are published for this system.; Risk assessment: 80.1 vs No per-category values are published for this system.. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Search & retrieval: Jev 1.13.0: 95.9; Super Decider: No measured result — Search and retrieval: scoring query-to-candidate relevance, reranking results, selecting useful context for RAG. 54 items (18 open, 36 sealed).Search &retrieval95.9 · n/aModel routing: Jev 1.13.0: 96.5; Super Decider: No measured result — Model routing: deciding which LLM or model tier should handle a prompt; classifying a request's intent, domain, difficulty or risk to pick a model. 132 items (131 open, 1 sealed).Modelrouting96.5 · n/aLLM guardrails: Jev 1.13.0: 67.0; Super Decider: No measured result — LLM guardrails: checking LLM inputs, outputs and tool calls: prompt injection, jailbreaks, policy violations, tool-call errors, answer quality. 339 items (209 open, 130 sealed).LLMguardrails67.0 · n/aLead generation: Jev 1.13.0: 88.1; Super Decider: No measured result — Lead generation: matching companies or inbound messages to an ideal customer profile, buyer intent, prioritising and routing sales leads. 98 items (61 open, 37 sealed).Leadgeneration88.1 · n/aCustomer support: Jev 1.13.0: 91.9; Super Decider: No measured result — Customer support: classifying tickets and customer intents, urgency, refunds, routing cases to a team, checking support replies. 389 items (176 open, 213 sealed).Customersupport91.9 · n/aInsurance claims: Jev 1.13.0: 67.5; Super Decider: No measured result — Insurance claims: first-notice-of-loss reports, claim complexity, missing information, claim triage. 31 items (14 open, 17 sealed).Insuranceclaims67.5 · n/aFinancial crime: Jev 1.13.0: 58.3; Super Decider: No measured result — Financial crime: suspicious transactions, KYC, fraud alerts, entity matching for investigations. 31 items (10 open, 21 sealed).Financialcrime58.3 · n/aLegal & compliance: Jev 1.13.0: 74.0; Super Decider: No measured result — Legal and compliance: contracts, policies, regulations, eligibility rules, checking documents against written requirements. 508 items (261 open, 247 sealed).Legal &compliance74.0 · n/aE-commerce: Jev 1.13.0: 89.8; Super Decider: No measured result — E-commerce marketplaces: product listings, product attributes, orders and deliveries, seller catalog policy. 71 items (48 open, 23 sealed).E-commerce89.8 · n/aRisk assessment: Jev 1.13.0: 80.1; Super Decider: No measured result — Risk assessment: estimating probabilities and severity of risks, incidents, vendor or underwriting risk. 78 items (39 open, 39 sealed).Riskassessment80.1 · n/a

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

The real-world application area of each decision, after the TypeSafe use-case map; an item can count in two. Chance-corrected competence per category (0 = chance, below chance negative, 100 = perfect), Jev 1.13.0: open+sealed; Super Decider: open+sealed items pooled; only categories with at least 30 items are spokes, smaller ones are listed below. Hover a category for its definition and item count.

Low sample, n < 30 — indicative only

Category (items)A: Jev 1.13.0B: Super Decider
Graphs and knowledge graphs (15)100.0 n=15—

No published value for either system (under 15 answered items, or no per-category values): Scientific discovery (1), Semantic code linting (4), Feature extraction for predictive modeling (7), Recruiting (3), Moderation and trust and safety (12), Advertising (2).

Not drawn on the radar: with so few items a single answer moves a category score by several points, so these values are noise-prone. These values become spokes once the item pool reaches 30 items per category.

What each category means · items per category
  • Search and retrieval — scoring query-to-candidate relevance, reranking results, selecting useful context for RAG. 54 items (18 open / 36 sealed)
  • Model routing — deciding which LLM or model tier should handle a prompt; classifying a request's intent, domain, difficulty or risk to pick a model. 132 items (131 open / 1 sealed)
  • LLM guardrails — checking LLM inputs, outputs and tool calls: prompt injection, jailbreaks, policy violations, tool-call errors, answer quality. 339 items (209 open / 130 sealed)
  • Lead generation — matching companies or inbound messages to an ideal customer profile, buyer intent, prioritising and routing sales leads. 98 items (61 open / 37 sealed)
  • Customer support — classifying tickets and customer intents, urgency, refunds, routing cases to a team, checking support replies. 389 items (176 open / 213 sealed)
  • Insurance claims — first-notice-of-loss reports, claim complexity, missing information, claim triage. 31 items (14 open / 17 sealed)
  • Financial crime — suspicious transactions, KYC, fraud alerts, entity matching for investigations. 31 items (10 open / 21 sealed)
  • Legal and compliance — contracts, policies, regulations, eligibility rules, checking documents against written requirements. 508 items (261 open / 247 sealed)
  • E-commerce marketplaces — product listings, product attributes, orders and deliveries, seller catalog policy. 71 items (48 open / 23 sealed)
  • Risk assessment — estimating probabilities and severity of risks, incidents, vendor or underwriting risk. 78 items (39 open / 39 sealed)

Not drawn: Other — none of the above: general reasoning, arithmetic or date puzzles, everyday assistant requests without a business workflow. 159 items (76 open / 83 sealed) — not a use case of its own, so it is counted but not drawn.

No items in this release: Gaming, Demand forecasting.

Competence per request type, open / sealed

Radar: competence per request type, open / sealed, two systemsCompetence per request type, open / sealed, Jev 1.13.0 vs Super Decider. Choice · open: 79.0 vs 82.9; Choice · sealed: 75.8 vs 81.3; Noul · open: 54.5 vs 84.0; Noul · sealed: 46.1 vs 84.0; Score · open: 63.3 vs 69.5; Score · sealed: 63.0 vs 71.6. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Choice · open: Jev 1.13.0: 79.0; Super Decider: 82.9Choice · open79.0 · 82.9Choice · sealed: Jev 1.13.0: 75.8; Super Decider: 81.3Choice ·sealed75.8 · 81.3Noul · open: Jev 1.13.0: 54.5; Super Decider: 84.0Noul · open54.5 · 84.0Noul · sealed: Jev 1.13.0: 46.1; Super Decider: 84.0Noul · sealed46.1 · 84.0Score · open: Jev 1.13.0: 63.3; Super Decider: 69.5Score · open63.3 · 69.5Score · sealed: Jev 1.13.0: 63.0; Super Decider: 71.6Score ·sealed63.0 · 71.6

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Chance-corrected competence for Choice, Noul and Score on the 300 open and 1200 sealed decisions.

Competence per tier — open set

Radar: competence per tier — open set, two systemsCompetence per tier — open set, Jev 1.13.0 vs Super Decider. Easy: 93.9 vs 97.7; Standard: 63.2 vs 77.2; Judge: 67.7 vs 75.0; Hard: 65.6 vs 79.5. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Easy: Jev 1.13.0: 93.9; Super Decider: 97.7Easy93.9 · 97.7Standard: Jev 1.13.0: 63.2; Super Decider: 77.2Standard63.2 · 77.2Judge: Jev 1.13.0: 67.7; Super Decider: 75.0Judge67.7 · 75.0Hard: Jev 1.13.0: 65.6; Super Decider: 79.5Hard65.6 · 79.5

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Per-tier competence, the three request types pooled by their published decision counts.

Competence per tier — sealed set

Radar: competence per tier — sealed set, two systemsCompetence per tier — sealed set, Jev 1.13.0 vs Super Decider. Easy: 78.4 vs 88.6; Standard: 68.7 vs 86.2; Judge: 66.9 vs 80.3; Hard: 58.9 vs 72.7. Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.−1000100Easy: Jev 1.13.0: 78.4; Super Decider: 88.6Easy78.4 · 88.6Standard: Jev 1.13.0: 68.7; Super Decider: 86.2Standard68.7 · 86.2Judge: Jev 1.13.0: 66.9; Super Decider: 80.3Judge66.9 · 80.3Hard: Jev 1.13.0: 58.9; Super Decider: 72.7Hard58.9 · 72.7

Linear signed scale: −100 at the centre, 0 on the bold middle ring, 100 at the rim.

Per-tier competence on the sealed decisions, types pooled the same way; item text stays private.
How the categories were made

Chance-corrected competence (0 = chance, 100 = perfect) over the category's items, open and sealed pooled, all tiers: per request type the method's competence cell (Choice and Noul accuracy above chance, Score 1 - error / chance error), then averaged over types weighted by item count. Not part of the JevBench Score; compare systems within a category, not categories with each other. Values can be negative (below chance).

Every one of the 1,624 decisions was labelled with one subject topic (the v1.2 topic list, unchanged) and one or two TypeSafe use-case categories (docs.typesafe.ai/concepts/use-case-map, plus Other) by Winnow-12B Q8 (the model behind System1 Models s1-pro) on our own GPU pod; 1,118 of the 3,248 calls were first made on s1-pro itself and agree 98.2 % with the pod run. Sealed items stayed on our own infrastructure. About 5 % of public items were checked by hand (Claude Opus 5.5); the item-group rules below fix the systematic misses found.

  • routing, routing_hard and tool_selection items (choose the model, agent or tool that handles a request) are Model routing first; the labeller tagged many by the request subject.
  • tool_guardrail, unsafe_action, adequacy, judge_hard and policy_compliance items (check an agent tool call or a drafted answer before it goes out) are LLM guardrails first.
  • lead_qualification items take the topic Finance & commerce (sales); the labeller read the scoring rubric as rules & law.
  • A second use case is kept when the labeller gave it probability >= 0.15 and it is not Other.
All values as a table
SpokeA: Jev 1.13.0B: Super Decider
The four score axes
Intelligence63.678.9
Calibration90.690.8
Speed91.285.3
Cost54.764.3
Capability by subject topic
Math & numbers40.5No per-category values are published for this system.
Coding & software80.3No per-category values are published for this system.
Rules, policy & law73.7No per-category values are published for this system.
Finance & commerce83.4No per-category values are published for this system.
Support & operations91.5No per-category values are published for this system.
Everyday language96.1No per-category values are published for this system.
Safety & security67.7No per-category values are published for this system.
Use cases (TypeSafe categories)
Search & retrieval95.9No per-category values are published for this system.
Model routing96.5No per-category values are published for this system.
LLM guardrails67.0No per-category values are published for this system.
Lead generation88.1No per-category values are published for this system.
Customer support91.9No per-category values are published for this system.
Insurance claims67.5No per-category values are published for this system.
Financial crime58.3No per-category values are published for this system.
Legal & compliance74.0No per-category values are published for this system.
E-commerce89.8No per-category values are published for this system.
Risk assessment80.1No per-category values are published for this system.
Competence per request type, open / sealed
Choice · open79.082.9
Choice · sealed75.881.3
Noul · open54.584.0
Noul · sealed46.184.0
Score · open63.369.5
Score · sealed63.071.6
Competence per tier — open set
Easy93.997.7
Standard63.277.2
Judge67.775.0
Hard65.679.5
Competence per tier — sealed set
Easy78.488.6
Standard68.786.2
Judge66.980.3
Hard58.972.7

Category radars count each answered item once from the pools named under each radar. API overlay rows use A4+P+L1+L2+L3; A5+P+L1+L2+L3; S+P+L1+L2+L3. Raw and unequated; cells under 15 answered items are omitted. Per-type and tier radars retain each row's original measurement pools: A4/A5 rows have 300 open plus 300 sealed items; full-set rows have S 1,200 plus P 300.

Languages

Raw chance-corrected competence per item language (0 = chance, 100 = perfect; can be negative), from each system's own measured items: Self-hosted rows use S 1,200 + P 300 + L1 (354 items) + L2 (33 items) + L3 (1,118 items), where answered. API rows re-run on A4/A5 use their 300/300-item sealed subset instead of S. Every row’s tag lists the pools it actually answered. L3 is a sealed language supplement (1,118 items), drawn 2026-10-07. Every language has at least 60 items in P ∪ L1 ∪ L2 ∪ L3. Header counts show the full S + P + L1 + L2 + L3 pool. Headline scores are unchanged; these raw cells are unequated, scored for language/category views only and outside the Composite. L3 items were written natively by Claude Sonnet 5.5, each solved blind and language-checked by GPT-6.1 Sol, with gold kept only when both agree or a second review confirms; no gold comes from Jev or any measured API. L3 is API-facing by design and is excluded from future headline draws. Rows with unfinished runs retain their actual coverage tags. L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of OpenAI rows (OpenAI Decisions, GPT-6 Luna, GPT-5.6 Luna) carry that exposure; headline scores do not use L3. C1 adds English items for thin radar categories (everyday language, safety and the other use case). L4 adds 20 new Hindi items for Fastino GLiNER 2.5 Decide only. They were authored with OpenAI and independently solved and language-reviewed with Anthropic before sealing. Seventeen L4 requests succeeded; three HTTP 400 failures remain in the scored observations and do not count as completed coverage. L4 is outside headline and category scores and the common header counts. L5 adds 60 new items for OpenJev DeBERTa v3 Large only: 20 each in Arabic, Hindi and Greek. They were authored with OpenAI and independently solved and language-reviewed with Anthropic before sealing, then checked against the original 512-token native context before keeper selection. All 60 native requests succeeded. Previous failures remain scored and do not count as completed coverage. L5 is outside headline and category scores and the common header counts. Cells under 15 completed supported responses are left empty; recorded input refusals count toward coverage. A dagger (†) marks fewer than 30 completed responses. Competence retains all supported scored observations, including failures and refusals. The mixed-language group is listed first, then English (1,306 items); the other 21 languages and the mixed-language group share 1699 items.

313 systems

JevBench v1.7.9 competence by item language and system
781306959088848383808077757575737372727171706965
Super DeciderAPI · not measured—818777†7686—76†—80†———————72†81†———66†
Myles-G 12BAPI · S+P (draw v1.6-regular-20261012)—566673†5986—58†—48†———————82†4†———43†
kettei-fatAPI · S+P (draw v1.6-regular-20261012)—809076†8687—81†—81†———————79†86†———78†
Fastino GLiDE (hosted API, 22 Oct 2026)API · S+P (draw v1.6-regular-20261012)—767478†7993—66†—47†———————70†49†———94†
Torchcast Decision 4B (v1.3)self-hosted · S+P (draw v1.6-regular-20261012)—667275†7168—82†—62†———————70†27†———66†
d1-ultraAPI · S+P (draw v1.6-regular-20261012)—706686†6789—63†—24†———————69†28†———44†
SubQ Snap 1.1API · S+P (draw v1.6-regular-20261012)—757576†8391—61†—39†———————47†77†———78†
Drex MaxAPI · S+P (draw v1.6-regular-20261012)—797285†8288—76†—54†———————71†70†———94†
Bespoke Nimble-v4-maxAPI · S+P (draw v1.6-regular-20261012)—717065†7382—71†—27†———————34†59†———61†
Super Decider (thinking)API · not measured—828180†6788—74†—78†———————68†81†———83†
SubQ SnapAPI · S+P (draw v1.6-regular-20261012)—746862†7284—67†—39†———————51†61†———66†
H2O-Lightning-4B v1.1self-hosted · S+P+L1+L2+L38563555960544938545841373446264946394855473123
Sage 1.3.0API · S+P+L1+L2+L38071485542494445575136444332264451656250593743
kettei-chubAPI · S+P (draw v1.6-regular-20261012)—627763†6183—77†—49†———————24†28†———72†
H2O-Lightning-31B v1.0.7self-hosted · S+P (draw v1.6-regular-20261012)—816675†8391—78†—65†———————75†28†———77†
Torchcast Decision 4B (v1.2)self-hosted · S+P (draw v1.6-regular-20261012)—606568†7678—71†—63†———————39†9†———74†
d1API · S+P+L1+L2+L38066325534443336644028304128143327485141471232
Mercury DecideAPI · not measured———————————————————————
Decisio 12B tuned commitself-hosted · not measured—607066†6183—55†—52†———————56†39†———71†
Chon-1.0-E4Bself-hosted · not measured—506875†6883—71†—45†———————61†25†———52†
decisio v0.8.0 on gemma-4-31B-itself-hosted · not measured———————————————————————
basal-omni-1.0-ultraself-hosted · S+P (draw v1.6-regular-20261012)—777670†6586—62†—58†———————71†39†———78†
Jev 1.13.0API · S+P+L1+L2+L3766838442248353436363015431173420364323471921
Microsoft-Decision-1 (Azure Foundry) — A2 re-runAPI · A2+P (draw v1.6-regular-20261012)—59——53†——————————————————
Quyet-1.0-Largeself-hosted · S+P+L1+L2+L38476475756665440665543525943296459676746554564
Celeris-1 DecisionAPI · S+P (draw v1.6-regular-20261012)—595951†5688—72†—43†———————52†11†———55†
Sieve-31B-v2self-hosted · not measured—716751†6791—51†—48†———————40†44†———61†
Torchcast Decision 4B (v1.1)self-hosted · S+P (draw v1.6-regular-20261012)—546265†7261—80†—46†———————33†0†———59†
Myles-G 31BAPI · S+P (draw v1.6-regular-20261012)—757171†6889—59†—40†———————76†38†———67†
Arcbright Elf 1 31Bself-hosted · not measured—726771†7486—59†—50†———————73†24†———77†
Jeff-1.1-Mediumself-hosted · not measured—758276†8894—85†—29†———————88†52†———78†
decider-12b v2self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8765515861404955614954435546435759675856405836
Xevn-4Bself-hosted · not measured—496372†4472—77†—56†———————37†25†———70†
Jev 1.14.0API · S+P (draw v1.6-regular-20261012)—637373†4275—51†—0†———————39†0†———44†
torchcast-decision-12bself-hosted · S+P+L1+L2+L38665496447483953645044474544335440685354445844
decider-12b v1, stock Gemma-4-12B-itself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8562485049424654615142464738324944615049405143
metask-jev-rain-31B-trainedself-hosted · S+P (draw v1.6-regular-20261012)—716968†6390—57†—46†———————73†36†———78†
vLLM-SR d3-miniself-hosted · not measured—526257†5673—80†—36†———————30†0†———28†
Bobcat 1.3self-hosted · not measured—697170†6387—62†—46†———————74†37†———66†
Microsoft-Decision-1API · not measured———————————————————————
wity-1 (Wity, reasoning always)API · S+P+L1+L2+L3757357683250334446432940173632294144554331185
wity-1 (Wity, reasoning auto)API · S+P+L1+L2+L38373596331602940333844333954241642563936451516
Chon-1.0-12B-Fastself-hosted · not measured—567175†7888—49†—49†———————71†31†———44†
Winnow-12B Q8self-hosted · S+P+L1+L2+L37864405939523954534541294433414636575534384440
deck-31Bself-hosted · S+P+L1+L2+L39178606955686560747156587150536860706159554560
Jeff-1.0-Largeself-hosted · S+P (draw v1.6-fastlane-20261009)99†718967†649156†—98†49†——————77†84†62†———59†
Cygnetself-hosted · S+P+L1+L2+L37957385641413239634534274236214245535434394640
Wald 26B-A4Bself-hosted · not measured—646568†7472—60†—36†———————58†29†———78†
decisio v0.8.0 on gemma-4-12B-itself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7956395539423338634129253435253744485638404227
attest-31B-v1.0API · S+P (draw v1.6-regular-20261012)—676774†5889—53†—45†———————82†26†———67†
DOPEST 1.0 31Bself-hosted · not measured—746075†6991—66†—51†———————79†42†———78†
Sage 1.4API · S+P (draw v1.6-regular-20261012)—656573†6583—67†—39†———————34†52†———61†
decisio v0.9.0 on gemma-4-12B-itself-hosted · S+P+L1+L2+L38156415538403141624128293634253744485637404331
Jev-Omniself-hosted · S+P+L1+L2+L38656385859444537585038274928255038575547484637
metask-jev-rain-12B-trainedself-hosted · S+P (draw v1.6-regular-20261012)—647481†7285—49†—45†———————59†25†———77†
Bobcat Flash 1.2self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8271454738543333514143373928264737485044582927
Xor 26B-A4Bself-hosted · S+P+L1+L2+L38061464048565136524434454332335040685050504246
SPX-CD FlashAPI · S+P+L1+L2+L3786338433640273449483122301783824375146501229
Messier One v0.4self-hosted · S+P (draw v1.6-regular-20261012)—516862†7072—65†—39†———————32†18†———55†
decider chat on Gemma-4-31B-itself-hosted · S+P+L1+L2+L37564585434535045565945395245324344514443444449
Hopper 12B trainedself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7954505142483339544232233828163632484734322434
metask-jev-rain-31B-trained service v3self-hosted · S+P (draw v1.6-regular-20261012)—746874†6382—55†—65†———————80†43†———78†
Surogate Rune 26B-A4B v3 (v1.6 pool)self-hosted · S+P+L1+L2+L37559474336473737454433374538195237554340502333
GEV-26B-Decideself-hosted · S+P+L1+L2+L37852333731423742372831233028213529422140282725
SPX-CD-Omniself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8053354046362932504441344426314822445635382434
Metask rain 4Bself-hosted · S+P (draw v1.6-fastlane-20261009)100†486574†606261†—30†8†——————0†41†0†———73†
shisa-de-1self-hosted · S+P (draw v1.6-regular-20261012)—456176†4588—52†—30†———————52†25†———50†
OpenAI DecisionsAPI · A5+P+L1+L2+L3L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of this row carry that exposure; headline scores do not use L3.7659364841552737433724234637303327403339374025
InstinctAPI · A4+P+L1+L2+L3675822353228273152482018341923223939394243137
Blink v0.3 26B-A4Bself-hosted · S+P (draw v1.6-regular-20261010-a2)—4654†90†457054†77†39†25———————39————0†
HS1 0.1.2API · S+P (draw v1.6-regular-20261012)—505473†4678—61†—35†———————58†37†———39†
baselabs elephant-1self-hosted · not measured—537472†6272—54†—32†———————70†24†———55†
Diffusion Jevself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.8259515749653741524448294345364944635050513816
René-1 31B FP8self-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7567453942464031553735375413254433474342462043
Amazon Strands Decider 26B-A4Bself-hosted · not measured—436849†4969—59†—56†———————32†15†———40†
Wald 4B v2.1self-hosted · S+P (draw v1.6-regular-20261010-a2)—5072†75†345543†54†54†49———————25————12†
pplx-decider-flash-v1.2self-hosted · S+P (draw v1.6-regular-20261012)—476665†5474—86†—24†———————0†0†———22†
Bespoke Nimble 9B v3self-hosted · S+P+L1+L2+L3806145503544303056412774032173623324260542037
Aplomb 1self-hosted · S+P+L1+L2+L3The headline keeps the original published measurement. Its originally pinned Hugging Face revision (f8e2dc81) was later deleted upstream, so the language and category supplements (L1, L2, L3) were measured on 12 Oct 2026 with the current upstream revision d6743291 (all file hashes recorded) on an isolated pod. That revision reproduces the original answers: 98 % agreement on the 300 public items (published bar 95 %) and 97.5 % on all 1,500 headline items. Each supplement carries the original P300 answers by stable item identity; they count once. All new (sealed) supplement items were answered; the only failed requests (3 public items, GPU out-of-memory) are covered by the original P300 answers. Headline, rank, Composite and pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.694643362641212244272214242811272725433338132
Respan Span-02self-hosted · S+P (draw v1.6-regular-20261012)—605057†6180—53†—37†———————70†29†———72†
Seb-9B-v2self-hosted · not measured—446574†3168—86†—33†———————19†5†———22†
wity-1 (Wity, reasoning off)API · S+P+L1+L2+L367523848293621261013171417176112430313333261
APUS-OpenJev-v1-9Bself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.7448333125371822374180202561628293034351712
Arcbright Elf 1 12Bself-hosted · S+P (draw v1.6-regular-20261012)—415360†5779—65†—30†———————42†0†———33†
Jeffan-31b-1710self-hosted · not measured—767158†6777—69†—45†———————76†58†———67†
Plumb-4Bself-hosted · S+P+L1+L2+L3724535453630382732342583240142431252827392220
SPX-CD ProAPI · S+P+L1+L2+L38270445537445343625035474119224737485429522947
Messier One v0.2self-hosted · not measured———————————————————————
spark-s1-4b-v6self-hosted · S+P+L1+L2+L380463331182891737352202223015292023102627
KnowLine 31B-Gen1self-hosted · not measured—817268†7588—63†—46†———————80†53†———67†
swanOneself-hosted · S+PThe pinned model weights cannot currently be accessed (Hugging Face returned HTTP 401). Supplement measurements remain pending access recovery; the radar shows the existing S + P items.—578668†4885†26†45†91†42†—————————————
AJev lm6qself-hosted · not measured—746876†6784—64†—43†———————77†59†———65†
Quyet-1.0-Mediumself-hosted · S+P+L1+L2+L360443332254225153532238152002933351921351214
Bespoke Nimble-v4API · S+P (draw v1.6-regular-20261012)—426751†3973—72†—15†———————22†0†———33†
Turion Jev 31Bself-hosted · not measured—757250†6283—68†—54†———————71†61†———72†
Vansa-3.4API · A4+P+L1+L2+L35447124228171210244311013191291726153335011
JPT-4Bself-hosted · S+P (draw v1.6-regular-20261012)—367563†2775—70†—30†———————11†14†———33†
levself-hosted · S+P+L1+L2+L35541282022221222282229822112141728192437537
NInfer Qwen3.8-Flash-Next mixedself-hosted · S+P+L1+L2+L37253415024333033553328164320273635543138432520
decider-4b v2self-hosted · S+P+L1+L2+L36242412511288222132320181318183222242911
baselabs mammoth-1self-hosted · not measured—777683†6590—65†—62†———————79†76†———61†
Jeffan-4B-2010self-hosted · not measured—396676†3063—83†—14†———————20†0†———22†
jev-localself-hosted · S+P+L1+L2+L37146433121312122333217720127212910342529115
GPT-6 Luna (low reasoning effort)API · A4+P+L1+L2+L3L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of this row carry that exposure; headline scores do not use L3.9796919391941009410099989694939694969710098969997
Autoloops – Gemma 4 31B ITAPI · A4+P+L1+L2+L37367445044566142616043484848476050554945534151
Wald 4Bself-hosted · S+P (draw v1.6-fastlane-20261009)95†435152†355451†—47†0†——————8†26†14†———15†
GPT-6 Luna (default medium reasoning effort)API · A4+P+L1+L2+L3L3 items were reviewed by an OpenAI model, so the L3-based language and category cells of this row carry that exposure; headline scores do not use L3.9797989691981009999969810096959895979995961009996
ClassOne Qwen 3.5 9Bself-hosted · S+P+L1+L2+L3Language and category cells use the unchanged original S+P observations and the newly completed frozen supplemental items (L1, L2, L3), answered with the original model and serving setup on an isolated pod. Each supplement carries the same original P300 answers by stable item identity; they count once in the union. Failed requests stay scored as failures and are not imputed. Headline, rank, Composite and original pricing are unchanged. L3 items were reviewed by an OpenAI model, so cells based on L3 carry that exposure; headline scores do not use L3.714538512636231930432301781411301718401331
JevK5 v0.3self-hosted · S+P+L1+L2+L36743404229192013283018028278162817223429318
metask-jev-4bself-hosted · S+P+L1+L2+L37340303415211619282412915409112316122640
deck-4B v1.0self-hosted · S+P+L1+L2+L367433745322221192930220312651323182736291917
GLiClass v4 9Bself-hosted · not measured—365865†5980—71†—45†———————14†0†———39†
Kahn1 4Bself-hosted · S+P+L1+L2+L3674036462835253029452553525211433192841302020
Clef-Flashself-hosted · S+P+L1+L2+L3694440318349194129231810201112232323361311
Decision-4Bself-hosted · S+P+L1+L2+L36143271903107202226261000211713152500
decisor-4bself-hosted · S+P (draw v1.6-fastlane-20261009)100†3670