Vera Largev1.7 · 10 s rule

Composite (secondary)

20.3 · Composite rank #111

Cost and measurement conditions

$0.217 per 1,000 decisions (estimated)

ESTIMATE (*est.), base-model market reference through the frozen pricing_v15 module (sha256 aa2afcd5a58d0c4cc0e3d6867d590c70b6ac58e971410c27faf9628f97bcf8b7, re-frozen 22 Oct 2026): exact base string Qwen/Qwen3.8-27B resolves directly to the listing USD 0.42/M input, USD 3.00/M output (dated 25 Sep 2026 catalog snapshots); applied here as qwen/qwen3.8-27b USD 0.42/M input, USD 0.0/M output, i.e. with the output priced at 0 under the pricing ruling on file. No explicit_ref override is needed. Estimated, not charged, not a vendor tariff or a quote from the author or any host (release-owner ruling on file). Tokens as reported by the usage block of the author's server: input_tokens counts the rendered question prompt, including chat template and options; output_tokens is the author's positive estimate sum(len(json.dumps(answer))//4) of the answer JSON; nothing is generated (pooling runner). Under the zero-output convention for native-probability rows (the zero-output pricing convention) the output is priced at USD 0/M; the server-reported estimate of 84,061 output tokens is disclosed but not billed; billing that estimate at USD 3.00/M would give USD 0.386660 per 1,000 decisions on the common basis = 5.99x the cap. Measured totals: 766,685 input and 84,061 estimated output tokens across the 1,485 answered items. Input usage equals the pre-registered first-party offline count item by item, offset 0 and 0 mismatches; this agreement does not turn the output estimate into a tokenizer count. The 15 refused HTTP 400 items carry no usage and count 0 tokens; these are the 15 standard long positions excluded from every row's cost on the common 1,485-item basis (v1.6.30), so common-basis totals are also 766,685 input and 84,061 estimated output tokens. COST ON THE COMMON BASIS: USD 0.216840 per 1,000 decisions on 1,485 items, 3.357 times the official cost cap USD 0.064595; estimated, not charged. OVER THE OFFICIAL COST CAP, DISCLOSED: ranked_over_cap; outside the official-caps views.

x2 + 0.15 s (assumption, not measured)

Evaluator-owned rented Verda GPU VM, one NVIDIA RTX PRO 6000 Blackwell Server Edition 96 GB (97,887 MiB), FIN-03 (Finland, EU), PAYG non-spot, instance type 1RTXPRO6000.30V, USD 2.5991917808219176/h recorded by the runner. Attempt 2 measured 24 Oct 2026 08:36-08:53Z, about USD 0.74 GPU rental (lead-reported approximation); teardown confirmed 08:53:47Z. Author server on 127.0.0.1:8200 in a --network none --read-only container; serial loopback driver. Read-only code, model, input and exported cache; empty credential variables, HF offline flags and author steering switches unset. Cut proof inside the container before held-out input: only loopback, no IPv4/IPv6 default route, DNS and TCP fail. Complete=true, deadline_hit=false, rc 0, measurement container 308 s (run cap 538 s), scored loop 155.156841 s over 1,500 requests. Frozen v1.6-regular-20261012 self-hosted input sha256 dfd36720077ef7c41ace367524896507c7f44ce71306fb3e659e8b85069d995b, checked inside the container. Raw output sha256 468809896014afa52e577106142dbf1ff21d4d75ee7b55fe02d1f5993ee8fcea (receipt only; not opened by this drafting worker). Final node-shred verified; exact-id teardown confirms provider absent and guard closed. GPU rental is separate from the benchmark's estimated base-model tariff.

Model and serving disclosure

THE AUTHORS' OWN SERVER: scar-ai/vera-large at cc5adb16238d9ebae887da71f994cd04963d251f with the author's vera-s1 0.1.6 package, installed from the repo's bundled wheel (sha256 94f5a09cf70e4480994230300963789841e0a7de3ff424b2df619fd76fe4de46) with --no-deps --no-index under a network-free image build; all members equal the PyPI 0.1.6 wheel's (different zip timestamps). Six author Python modules; the model repo's separate Python files and fp32 branch were not fetched. Author files are unchanged; DISCLOSED SERVING WRAPPER: a one-line wrapper around vera.load passes max_num_seqs=256 through the author's own **engine_kw API, then runs vera.serve as __main__, with --model /model --engine vllm --max-model-len 32768 --host 127.0.0.1 --port 8200. This caps the scheduler's maximum co-batched sequences, following the pre-sealed attempt-1 startup failure; it does not change the author's 32,768-token limit, prompt rendering or refusal set. vLLM 0.30.0 pooling runner (token_embed), bidirectional full attention (is_causal: false), causal Gated-DeltaNet layers, prefix caching and chunked prefill off, max_num_batched_tokens = max_model_len = 32768, author-default gpu_memory_utilization 0.9, fp32 option-pointer readout. One request at a time. Image: vllm/vllm-openai:v0.30.0@sha256:8a69ffad015f138d7170c4ddc429e230a3bc1c1719f67e14324749df200a4b90 plus the one wheel, derived image sha256:9ab6ff921642238843ef1daa85e6c05cf38c938cb7f776e4c84a12bcd6d6b61c. torch 2.13.0, transformers 5.17.0, tokenizers 0.23.2, safetensors 0.8.0. The author's export_backbone transformed the pinned backbone offline before any held-out input, then the cache was mounted read-only. Export verification: 850 bf16 backbone tensors, names/shapes/dtypes checked, 43 tensors byte-compared by sampling, every exported file hashed, verdict OK; not a byte comparison of every tensor. Weights: 25,628,527,110 parameters (25,624,600,064 bf16 backbone + 3,927,046 fp32 heads), 10 shards totalling 51,265,014,328 bytes, 16 data files in the pinned manifest (sha256 57d88387c7503633362b61eb9c17d05ea8608accf48774ab13cf68f51634cdde). The submitted code and serving wrapper were reviewed and cleared before the run; evaluator exposure and the device were cleared for both attempts. The source, runner, device selection change and serving wrapper changes were reviewed before the run; required safeguards were checked and the code and device cleared. DISCLOSED PATTERNS: dormant Hub downloads and Hub-capable loaders, weights-only torch.load fallbacks absent from the safetensors serving path, steering environment switches asserted unset, and the contained pre-input cache-writing export. No subprocess, outbound HTTP, telemetry, credential reads or request-body logging in the reviewed author package; offline flags and the network cut were checked inside the measurement container. FIRST MEASUREMENT of Vera Large in this intake. AUTHOR'S OWN SERVER on our rented hardware, vLLM pooling path with the disclosed max_num_seqs=256 wrapper; author files unchanged. NATIVE OUTPUTS: choice and score answers with probabilities, and noul P(true), from the author's readout and option pointer; probabilities rounded to 4 decimals; no generated explanation. AUTHOR EXPOSURE: training and contamination statements in the card are author claims; no independent overlap or contamination audit was run by us. The card's Decision Index 68.4 on a 5% sample and 139 ms median on an AMD MI300X are self-reported and do not transfer to this measurement. LICENCE NOTE: Apache-2.0 in the model card and package metadata; no standalone LICENSE file. ATTEMPT HISTORY: attempt 1 (pod window 24 Oct 2026 06:22-06:32Z, about USD 0.45) failed BEFORE any held-out item, at vLLM startup. CUDA-graph memory profiling tried to allocate a 78,920,024,064-byte (78.9 GB) throwaway cache of 512 sequences alongside about 51 GB of weights. pre-registration Amendment 3 documents the fix: max_num_seqs=256 through vera.load(**engine_kw), reducing that profiling allocation to 39,460,012,032 bytes; author files, the 32,768-token limit and the refusal set unchanged. The runner was rechecked, and code, evaluator exposure and device clearance preceded attempt 2, this measurement. NOT ANSWERED, counted wrong: exactly the 15 pre-registered long items exceed the author's 32,768-token limit (HTTP 400), at positions in the pre-registered long-item set; never truncated. Answered: 1,485 of 1,500 items sent, HTTP 200; all observed refusal ids and positions match the preregistration, no unexpected failures. 10 s RULE: 0 decisions over 10 s, so no decision is scored wrong under the 10 s rule in the v1.7 share view (the frozen v1.6 and v1.7-off views apply no time limit); maximum 0.294317 s (median 0.092944 s, p95 0.161959 s) over the 1,500 requests. DETERMINISM: 50 positions repeated, 50/50 bit-identical, 0 argmax flips, maximum probability difference 0.0, on the server's rounded outputs. CONTEXT: author limit 32,768 tokens; no truncation. OVER COST CAP: the base-model estimate (input priced, output at 0 under the zero-output convention, the pricing ruling on file) is above the cost cap, so the row is ranked_over_cap (uncapped views only); the exact figure and ratio are in the cost basis.