HS1 0.1.2 (Hyland)

Data: JevBench v1.6.40, published 2026-10-22

Capability 65.0 · API offering, ranked on the API leaderboard

Last measured: 2026-10-21 · measurement release: v1.6.36

Proprietary hosted API (no weights release); declared base Gemma 4 26B-A4B-it is under the Gemma 4 licence

Base model: google/gemma-4-26B-A4B-it (self-reported) · Customer statement (order notes, 21 Oct 2026): a LoRA fine-tune of Gemma-4-26B-A4B-it served in FP8. Not publicly documented and not independently verified.

intelligence

53.6

calibration

76.5

speed

93.7

cost

49.1

Composite (secondary)

61.5 · Not ranked on this board

Cost and measurement conditions

$0.050 per 1,000 decisions (estimated)

ESTIMATE, v1.5-M2 base-model reference rule. The customer states a benchmark tariff of USD 0.056 per million input tokens, output free (one NVIDIA L40S, AWS on-demand); there is no public price page for it, so it is not a published list price. Under the method, the base-model reference price applies as an ESTIMATE for this vendor-hosted row, as for attest-31b-v1.0: base model google/gemma-4-26b-a4b-it through the frozen pricing_v15 module (sha256 1f4d1f7b1ce97ab27b938043444b9c2757e4a9dbecc799a803122588401f5bf1) reference() -> google/gemma-4-26b-a4b-it on OpenRouter, USD 0.09 per million input tokens, USD 0.30 per million output tokens. The customer-stated tariff is shown for information and is not used for the Cost axis. Applied to this row's own provider-reported tokens, first answer per item (2,012,340 input and 0 output tokens over the 1,500 items; 820,650 input and 0 output tokens over the 1,485 items of the common cost basis; the 15 very long items count for no row's cost, whether the row refused or answered them). Estimated, not a vendor bill and not a quote. COMMON COST BASIS (v1.6.30): cost per 1,000 decisions is computed on the common 1,485 items of this draw, leaving out the 15 very long items (at least 76,358 input tokens as counted by the rows that answered them in full).

p50 latency: 0.2 s

none (API)

Paid public fast-lane order 880b09e8. Vendor-hosted Hyland API (customer-supplied HTTPS endpoint, model `hyland-hs1`), through the unchanged TypeSafe adapter; latency measured from Sandy (Germany) over the official per-item latency subset of the first-occurrence rows, without API adjustment. Full S1200+P300 draw of v1.6-regular-20261012 through the unchanged run_api_full_v161.py; the full-set exposure gate logged all 1,200 sealed items to provider `hyland` (no proxy) before the first request. 1,500 of 1,500 unique items answered with HTTP 200 and the served model was `hyland-hs1` on every row; option-letter log-probability readout, nothing generated (0 output tokens). Run with the customer's own API key; hosting implementation and deployed weights were not independently inspected.

Published source