H2O-Lightning-31B (image)

Cost and speed evidence

Cost per 1,000 decisions

USD 0.075275

Cost basis

measured GPU seconds x $1.19/GPU-hour

Measured latency · p50 / p95

0.219 s / 0.280 s

Adjusted latency · p50 / p95

0.588 s / 0.710 s

Latency adjustment

v1.4 local latency adjustment

Public accuracy

81.3%

Sealed accuracy

84.8%

Measurement note

Official ImageJev v0.3 S1200+P300 pass 1 on 22 Oct 2026 (1,500 items, the same self-hosted draw as the other GPU rows), paid fast-lane evaluation published under the order's public-visibility setting. Model: h2oai/h2o-lightning-31b at revision ca6e6f6e8e08e2d21c8c9f45261e05444626647a, base google/gemma-4-31B-it. Served using the author's custom vLLM 0.30.0 build and documented one-engine profile, FP8 weights (online quantization of the bf16 checkpoint), one RTX PRO 6000 Blackwell, offline after weights and code were hash-verified. Request: one System One choice question with the item question as state, the item image as base64 and the same options. The one-engine profile reads each image once at a 560-token budget; its image re-read is disabled (the separate decision-index profile enables re-reading). 1,500 of 1,500 answered, no inference errors, no retries. Latency and billed GPU seconds are the wall clock of each loopback request, one request at a time (shim and server overhead included), as for the H2O-Lightning-4B image row; other self-hosted rows generally use CUDA-event adapter timing. TypeSafe-20 was not run in this pod session; a follow-up adds those cells.

Published 2026-10-22. Scores and ranks can change in a later release. See the method notes.