JevBench v1.5.7 · individual system
Open-Jev 27B v1.1 (Zefan Cai, LoRA + decision head on Qwen3.8-27B)
Jev rebuild · by Zefan Cai · Openness: open weights
Base model: Qwen/Qwen3.8-27Bsourcev1.5 roster addendum A8
Apache-2.0
JevBench v1.5.7 score
2.927
Option A: rank #76 of 111 ranked systems.
The three option scores and ranks are published independently; the headline is Option A.
| Option | Score | Rank |
|---|---|---|
| A · headline | 2.927 | #76 of 111 |
| B | 3.192 | #75 of 111 |
| C | 2.927 | #74 of 111 |
Published axes
- intelligence
- 60.7
- calibration
- 80.2
- speed
- 71.0
- cost
- 14.4
Bands use the published 0–100 axis values; the marked reference is Jev 1.13.0 when that axis is available.
Run and cost evidence
- Run status
- complete · 1,624 decisions · 0 missing
- Cost
- $0.716 per 1,000 decisions · estimate · Frozen BASE_REFERENCES Qwen/Qwen3.8-27B market price (USD 0.42/M input, 3.00/M output) applied to 2,767,510 input tokens and zero output tokens over 1,624 decisions; self-hosted estimate, not measured hosted billing. Lineage verified by architecture; no pricing gate. The author method encodes each candidate independently.
- Median latency
- 1.550 seconds, adjusted · x2 + 0.15 s (assumption, not measured)
- Endpoint condition
- Self-hosted author loader, Transformers reference path on RTX PRO 6000 Blackwell; speed is a LOWER BOUND: flash-linear-attention and causal-conv1d kernels were not installed. The author documents the kernel path as answer-identical. Each candidate is encoded independently, producing high input-token usage.
- Published source
- https://huggingface.co/ZefanCai/Open-Jev-27B-v1.1
Model and serving disclosure
Qwen3.8-27B (lineage verified by architecture)
Values come from the public v1.5.7 aggregate. Scores and ranks may change in a later release.
Read the full leaderboard, the v1.5.7 release page, and the published method.