{"schema_version":1,"total":1,"offset":0,"limit":100,"observations":[{"id":"public:9a01d34c2c07108bef0d42d9","benchmark_id":"context-arena-mrcr-v2::8-needle","subject":{"source_id":"tencent/hy3","name":"tencent/hy3","model_id":"hy3::default","variant":null,"harness":null},"value":26.94049033136195,"unit":"percent","basis":"derived","source":{"url":"https://contextarena.ai/api/needle-summary?needles=8","retrieved_at":"2026-09-21T17:13:26.665692+00:00","published_at":null,"file":"data/raw/benchmarks/daily-evidence/2026-09-21-context-arena/949e2c8ad74e8a238d21.gz","sha256":"949e2c8ad74e8a238d21390d033bd16190cb27ca11d764d818f2e0f1040b0701","locator":"context_arena_summary; source row 176; tencent/hy3; field cum_avg_128k"},"protocol":"Context Arena (Dillon Uzar) runs Google DeepMind's MRCR v2 (multi-round coreference resolution, 8 needles, the \"full\" test set) against each model through its API: a long synthetic conversation holds eight identical writing requests, and the model must reproduce the i-th response, prefixed by a random hash, scored by string similarity (missing hash = 0). Tests are grouped by total token count into power-of-two bins. The value is the cumulative average up to 128k — the unweighted mean of the 8k, 16k, 32k, 64k and 128k bin scores, the \"upto_128K (cumulative)\" figure GDM reports — in percent. Context Arena's AUC @128k/@1M, the 1M cumulative average, the per-bin scores and test counts, total runs and the latest run date stay in each row. Rows the site marks insufficient or unranked, and rows whose bins up to 128k are incomplete, are not scored. The site says results come from running models against the published GDM dataset via API; it does not state whether tools were available, which GDM's README asks every MRCR report to state. Secondary benchmark, never a Composite input.; source row: {\"model_slug\":\"tencent/hy3\",\"reasoning_mode\":null,\"provider\":\"Tencent\",\"run_providers\":[\"Tencent Cloud\"],\"max_context_length\":262144,\"deprecated\":false,\"total_runs\":484,\"bins_upto_128k\":{\"8192\":[0.484219,80],\"16384\":[0.344489,80],\"32768\":[0.188698,136],\"65536\":[0.179791,85],\"131072\":[0.149828,103]},\"auc_128k\":0.2001996383627551,\"auc_1m\":0.03308359112750438,\"cum_avg_1m\":0.16837806457101218,\"has_incomplete_bins\":false,\"latest_run\":\"2026-08-13T02:54:29.254Z\"}","comparison_key":null,"source_basis":"measured","derivation":{"formula":"Source value × 100 to registry units","inputs":[0.2694049033136195]},"supporting_sources":[{"url":"https://raw.githubusercontent.com/google-deepmind/eval_hub/master/eval_hub/mrcr_v2/README.md","file":"data/raw/benchmarks/daily-evidence/2026-09-21-context-arena/7b2010f4685643f4af7b.gz","sha256":"7b2010f4685643f4af7bdccfb5e58e6822edea23d009d75cda4a86fa5c558f43","retrieved_at":"2026-09-21T17:13:30.878266+00:00","published_at":null,"locator":"method_source"}],"join_note":"Reviewed identity map 2026-09-21: label names hy3 without a setting; the catalog has exactly one configuration, the default (hy3::default)"}],"divergences":[],"cell":null,"coverage":{"model":null,"benchmark":{"total_models":924,"available":47,"unknown":877,"not_tested":0,"not_published":0,"source_unreachable":0,"contested":0,"measured":47,"self_reported":0,"observations":183,"unmatched_observations":136}},"collection":{"benchmark_id":"context-arena-mrcr-v2::8-needle","status":"collected","source_url":"https://contextarena.ai/api/needle-summary?needles=8","reason":"183 source results parsed; configurations remain separate; unmatched model identities are retained."},"coverage_note":"Denominators are catalog model configurations × versioned registry entries; capability_available excludes Efficiency (cost) boards. Basis counts count covered cells, not runs; measured and self_reported may overlap. Unmatched source identities remain in observations and never inflate catalog coverage."}