diff --git a/BLOCKED.md b/BLOCKED.md index b8d77a3f..c24cbdc8 100644 --- a/BLOCKED.md +++ b/BLOCKED.md @@ -1,5 +1,14 @@ # BLOCKED +## 生时校正验证:独立盲测资格与申报偏差真实分布(2026-09-20) + +- v3 / v4 的案例成绩已曝光。`BUG-427/428` 禁止将已见案例重新算作独立盲测,任务书没有授权推翻;固定实现、重跑或刷新哈希不能恢复未见性。本轮 T2 只能提供固定口径重跑,独立发布验证仍 blocked,确认门保持 `not_ready`。 +- 真实用户的申报偏差分布目前不可得:需要有独立出生记录的受控用户样本和合规采集授权,目前无受控账号;不读取、猜测或借用账号。本轮 T1 只能回答给定偏差时的敏感性,不能推算总体准确率或现实失败比例。 +- 本机无项目 `.venv`,Python 3.11.7 的 mcp/hypothesis 缺失;快速门及历史 v2 封存哈希断言的基线对照见 `docs/tasks/PROGRESS-rectification-validation-20260920.md`。不改历史封存、不削弱断言来换通过。 +- 前端两侧构建均因 Turbopack 不接受指向工作树外的 node_modules junction 失败;Static/gzip 无完整产物。类型检查和 lint 通过不等于构建通过,补验清单见 `docs/testing/rectification-validation-20260920.md`。 +- 独立审查发现生产矩阵的 transition proximity 仍对跨日候选复用单一出生日期。本轮仅在新增离线评测按日期分组适配并回归,生产实现受本单“不改打分”红线保护,未修;因此本轮跨日结果不是未经改动的生产端到端回放。生产是否另立修复单由产品决定。 + + ## ~~私人案例清理:远程 review 分支推送认证失败(2026-09-19)~~ - **已解除:** 恢复认证后已成功推送 `codex/owner-case-purge-20260919`,远端 SHA 与本地提交一致;未推 staging 或 main,未部署。此前失败历史保留,不再代表当前状态。 diff --git a/docs/BUG_HISTORY.md b/docs/BUG_HISTORY.md index 433f757b..6abf3866 100644 --- a/docs/BUG_HISTORY.md +++ b/docs/BUG_HISTORY.md @@ -12870,3 +12870,52 @@ - 相关记录:BUG-970 - 复发自:无 - 修复版本:`8dc460da` + +## BUG-978 | 封存契约当前打分身份与资料审计状态过期 + +- 状态:resolved/partial +- 首次发现:2026-09-20 +- 最近更新:2026-09-20 +- 影响面:封存 holdout 契约、当前实现评测身份与离线成绩归属。 +- 用户现象:历史冻结值、历史 sidecar 当前值、当前代码实测哈希互不相同;契约仍写旧资料审计状态与评测日期,当前成绩无法正确归属。 +- 触发条件:打分文件发生变化而契约未刷新,再读取契约判断当前引擎验证状态。 +- 根因:人工维护契约;既有测试仅与旧报告相互核对,没有断言真实当前树身份与数据集状态。 +- 修复:本轮新增固定口径重跑与 freshness 回归;执行与证据见 `docs/tasks/PROGRESS-rectification-validation-20260920.md`。保持发布六键与确认门不变。 +- 验证:freshness 与确认门相关定向通过;独立复算逐例结果及汇总与冻结重跑 JSON 一致,六个 runtime 键未变。新增测试通过既有快速门 glob 桥接实际收集。旧 v2 哈希测试与 quick 环境失败与基线相同,详见进度。独立盲测因案例曾曝光仍 blocked,固定实现重跑不恢复未见性;resolved 仅指元数据过期与可追溯缺口,partial 指新独立盲测未完成。 +- 防复发:当前打分文件 SHA-256、源审计状态与契约必须一致;新鲜度检查须进入实际快速门收集范围,不能仅存在于测试目录。新盲测还必须有未曝光样本和独立标注审核。 +- 相关记录:BUG-098、BUG-427、BUG-428。 +- 复发自:BUG-098 的独立 holdout 边界与 BUG-427 的身份绑定缺口;前者要求未被落实为独立封存流程,后者没有当前树 freshness 检查,旧 sidecar 自洽仍可过测试。 +- 修复版本:本轮执行分支,未交付。 + +## BUG-979 | 真值中心候选窗掩盖申报偏差导致的真值出窗 + +- 状态:resolved +- 首次发现:2026-09-20 +- 最近更新:2026-09-20 +- 影响面:离线候选窗评测与生产申报中心搜索窗的可比性。 +- 用户现象:离线评测真值永在候选窗中心,无法衡量真实申报偏差大于搜索半径时的失败模式。 +- 触发条件:用户申报偏离独立记录且偏差超过候选窗半径。 +- 根因:旧评测从真值构窗,缺少独立的申报偏移变量和真值在窗覆盖指标。 +- 修复:并行新增偏移敏感性评测,不修改历史候选函数;排名后才揭示真值,用既有 opaque 哈希破同分。 +- 验证:完整修正版 900 组合、45 格,排除 0;新增偏差与 freshness/桥接回归 28/28 通过(包含桥接重复收集),真实引擎跨日分组与逐候选重算相同。报告见 `docs/research/reported_offset_2026_09_20.md`,口径与哈希见同名 JSON。resolved 仅指离线缺少偏差维度的缺口,不代表产品分钟准确或生产跨日问题已修。 +- 防复发:锁定零偏移、窗外、口径字段、非真值破同分与跨日行为;窗口覆盖与排序命中分开记录,不以敏感性代替真实用户准确率。 +- 独立审查追加:首版偏差评测的矩阵为全窗共用一个日期,跨日时辅助 transition proximity 有偏差。已在离线适配层按候选日期分组计算并与逐候选真实引擎结果对照,不修改生产打分;原初扫统计作废。生产同类日期处理不在本单修复范围,记入 BLOCKED,不能声称生产端到端等价。 +- 相关记录:BUG-098、BUG-978。 +- 复发自:BUG-098;既有防复发聚焦候选公开门与同案稳定性,没有离线与生产窗心可比性的测试。 +- 修复版本:本轮执行分支,未交付。 + +## BUG-980 | 缺少事件丰富且从未曝光的独立封存集 + +- 状态:investigating +- 首次发现:2026-09-20 +- 最近更新:2026-09-20 +- 影响面:分钟级评测可推广性与独立发布证据。 +- 用户现象:旧封存集事件少;事件丰富的开放集用于调参、成绩已见,不能替代新的独立验证。 +- 触发条件:把开放评价集或真值方向最佳答案回放当成产品分钟级准确率。 +- 根因:开放集建设之后未建立独立采集、未曝光、独立人审的封存孪生集。 +- 修复:本轮仅制定 v5 采集协议与不重叠公开候选池,不采事件、不生成新成绩。 +- 验证:协议含事件/领域目标、独立来源、人审封存、曝光日志、工作量和发布边界;备选池与旧集去重通过(仅候选资格筛查,原文准入仍需复核)。v5 未建成,保持 investigating。 +- 防复发:采集、标注、评分权限分隔,调参前封存,首次揭晓前不得以事件内容调参;拒绝自动置真人审标志和重新封存已曝光案例。 +- 相关记录:BUG-098、BUG-428、BUG-978、BUG-979。 +- 复发自:BUG-098;原记录明确延期独立分区与校准,本轮仍无真正独立样本,不能将文档协议写成验证完成。 +- 修复版本:协议执行分支;数据集未就绪,不标 resolved。 diff --git a/docs/research/ACTIVE_FRONTS.md b/docs/research/ACTIVE_FRONTS.md index d25b007f..4edd4432 100644 --- a/docs/research/ACTIVE_FRONTS.md +++ b/docs/research/ACTIVE_FRONTS.md @@ -2,6 +2,14 @@ This file is the small index for the current engineering fronts that still drive code changes. +## 生时校正验证完整性(2026-09-20) + +- [申报偏差敏感性](reported_offset_2026_09_20.md):申报中心搜索窗与真值出窗,不能当真实用户准确率。 +- [固定口径重跑与契约](sealed_holdout_rerun_2026_09_20.md):已曝光 v3 的当前实现成绩,不是新的独立盲测;确认门关闭。 +- [v5 独立封存采集协议](sealed_holdout_v5_protocol_2026_09_20.md):本轮只出协议,不采事件。 +- 前置结论:[分钟分辨率两轮研究](rectification_minute_resolution_closure_2026_09_14.md)。不重走已证伪的加权重与簇合并假设。 +- 执行状态:[本轮进度](../tasks/PROGRESS-rectification-validation-20260920.md)。 + ## Shortest-Path Closure Order (2026-06-29) To avoid scope drift, current work stays inside these four closure lanes only: diff --git a/docs/research/pre_work_error_ledger.md b/docs/research/pre_work_error_ledger.md index d2caf80a..8c6b8833 100644 --- a/docs/research/pre_work_error_ledger.md +++ b/docs/research/pre_work_error_ledger.md @@ -2,6 +2,13 @@ Purpose: read this file before substantial project work. It exists to stop repeat mistakes caused by multiple Codex windows, WorkBuddy mirrors, local drafts, backup folders, and partial cloud-git visibility. +## 2026-09-20 · 校正验证补缺的本机验收复现 + +- 基线与实现工作树均使用 Python 3.11.7(无项目 `.venv`;`python3` launcher 退出 49)。开工预检远端 verified,但同一碎片镜像路径断言失败;不得把同步成功写成预检通过。 +- quick 两侧均在 `interpretation_source_inventory_gate.py` 退出 1,缺 `mcp`;两侧输出的 46 条缺引用消息逐项相同,不冒写快速门通过。历史 v2 封存哈希测试在基线已失败,未改历史记录或冻结文件消除它。 +- 前端构建两侧均因外部 node_modules junction 被 Turbopack 拒绝,不能出具 Static/gzip 验收。全量并发测试存在文件级资源失败,需保留基线逐项对照与定向复验,不以较少失败数宣称全部通过。 +- 防复发与补验见 `docs/tasks/PROGRESS-rectification-validation-20260920.md`、`docs/testing/rectification-validation-20260920.md`。本轮未修改环境权限、依赖、workflow 或运行主仓。 + ## 2026-09-19 · 私人资料清理的索引依赖预检缺口 - 本轮基线 `35ca7299c`。删除前独立审计发现:主 Skill/MCP 入口无直接引用,不等于台账无读者。证据包索引仍登记全部拟删台账,完整性检查会验证目标路径;派生视图和研究矩阵也保留引用。原清单直接删除会造成缺失路径回归,初期因此暂缓;产品补充授权后已完成精确删除与引用修复,索引缺失路径仍为零。交付状态见本轮进度记录。 diff --git a/docs/research/reported_offset_2026_09_20.json b/docs/research/reported_offset_2026_09_20.json new file mode 100644 index 00000000..fb85c9f8 --- /dev/null +++ b/docs/research/reported_offset_2026_09_20.json @@ -0,0 +1,12201 @@ +{ + "scope": "reported_offset_sensitivity_not_product_accuracy", + "specification": { + "ayanamsa": "raman", + "node_mode": "mean", + "radii_minutes": [ + 15, + 30, + 60 + ], + "offsets_minutes": [ + -30, + -20, + -15, + -10, + -8, + -5, + -3, + 0, + 3, + 5, + 8, + 10, + 15, + 20, + 30 + ], + "minute_step": 1, + "dataset": "references/real_case_calibration/minute_rectification_holdout_v3.json", + "dataset_sha256": "45aa76ab79dfc6c6404b6252083e1aa5c7e2e2b7de811fc11042a8ad2da3f0ea", + "dataset_benchmark_id": "minute_rectification_fact_ranker_v4_holdout_v3", + "implementation_sha256": "b15d9ea15227cd58b3097555537fa7c37fb5a993905ada4f5aa04f52de1f8f18", + "implementation_sha256_prefix": "b15d9ea15227cd58", + "production_scoring_files": [ + "scripts/active_rectification_event_engine.py", + "scripts/active_rectification_events.py", + "scripts/ashtakavarga.py", + "scripts/dasha_analyzer.py", + "scripts/divisional_charts_extended.py", + "scripts/domain_calculation_service.py", + "scripts/jaimini.py", + "scripts/minute_rectification_fact_ranker_v4.py", + "scripts/minute_rectification_feature_facts_v4.py", + "scripts/narayana_dasha.py", + "scripts/rectification/candidate_contrast.py", + "scripts/rectification/case_holdout.py", + "scripts/rectification/contracts.py", + "scripts/rectification/dasha_transition_proximity.py", + "scripts/rectification/event_probes.py", + "scripts/rectification/scoring_service.py", + "scripts/shadbala.py", + "scripts/varga.py" + ], + "production_scoring_sha256": "115c3fbcffdaff49aa7a2ef0fac88eba6e7ac6056641f94d933c8061701e73e6", + "research_files": [ + "scripts/research/reported_offset_sweep.py", + "scripts/research/cluster_width_lib.py", + "scripts/research/probe_supply_after_six.py", + "scripts/research/sealed_holdout_rerun.py", + "scripts/minute_rectification_blind_eval.py" + ], + "research_implementation_sha256": "8d0c613dc0900adbe798c8bd0bdd001b5ae26a8cfe731e6cccf8207f39e7d370", + "hash_scope": "explicit_identity_file_sets_not_a_transitive_dependency_lock", + "scorer": "native_event_contribution_matrix_not_shadow_fact_ranker", + "delivery": "initial_signature_clusters_peak_gap_lt_8_envelope_no_answers_no_elimination", + "rank": "score_desc_then_sha256(benchmark_id:case_id:candidate_time)", + "grid": "every_minute_inclusive_not_production_two_minute_sampling", + "cross_midnight": "date_aware_candidates_distance_and_matrix_grouped_by_candidate_date", + "evaluator_sha256": "87f87859bb399da1e860ab5e750e68088401a5a82fbba10ba709c17e5266167c", + "replay_revision": "candidate_date_grouped_v2", + "supersedes": "initial_sweep_invalidated_cross_midnight_transition_anchor", + "is_blind_evaluation": false, + "truth_hidden_from_ranker": true, + "results_previously_seen": true, + "must_not_use_for_tuning": true + }, + "excluded_cases": [], + "case_count": 20, + "trial_count": 900, + "trials": [ + { + "case_ordinal": 1, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 36.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 35.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 7.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 26, + "top_1_hit": false, + "top_1_minute_error": 7.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 7.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 7.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 15.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 18.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 19, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 1, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 33, + "top_1_hit": false, + "top_1_minute_error": 36.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 36.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 15.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 46, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 42, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 40, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 40, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 60 + }, + { + "case_ordinal": 1, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 58 + }, + { + "case_ordinal": 1, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 19, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 1, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 118 + }, + { + "case_ordinal": 1, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 1, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 42, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 19, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 33.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 2, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 46, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 48, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 48, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 42, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 40, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 33, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 33, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 49.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 49.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 2, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 77, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 80, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 81, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 82, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 84, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 86, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 86, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 86, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 84, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 85, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 85, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 85, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 85, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 81, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 2, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 71, + "top_1_hit": false, + "top_1_minute_error": 24.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 12.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 19 + }, + { + "case_ordinal": 3, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 24 + }, + { + "case_ordinal": 3, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 29 + }, + { + "case_ordinal": 3, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 3, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 3, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": false, + "candidate_count": 61, + "delivery_width_minutes": 44 + }, + { + "case_ordinal": 3, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 53, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": false, + "candidate_count": 61, + "delivery_width_minutes": 54 + }, + { + "case_ordinal": 3, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 3, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 111, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 10.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 4, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 4, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 4, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 11, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 9, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 8, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 5, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 26, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 25, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 25, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 17, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 5, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 5, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 9, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 6, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 59.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 25, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 33, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 40, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 54, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 6, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 78, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 82, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 82, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 63.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 83, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 82, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 77, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 6, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 69, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 15.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 16, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 17.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 7, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 17, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 19, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 60.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 7, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 86.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 80.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 57.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 60.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 44, + "top_1_hit": false, + "top_1_minute_error": 60.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 46, + "top_1_hit": false, + "top_1_minute_error": 64.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 49, + "top_1_hit": false, + "top_1_minute_error": 64.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 64.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 56, + "top_1_hit": false, + "top_1_minute_error": 64.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 61, + "top_1_hit": false, + "top_1_minute_error": 80.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 7, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 71, + "top_1_hit": false, + "top_1_minute_error": 84.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 26, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 15.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 10.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 17, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 8, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 56, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 50, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 48, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 8, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 8, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 103, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 35.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 5, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 45.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 9, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 25, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 35.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 27.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 5, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 4, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 9, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 70, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 60, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 55, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 50, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 48, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 119 + }, + { + "case_ordinal": 9, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 40, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 116 + }, + { + "case_ordinal": 9, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 113 + }, + { + "case_ordinal": 9, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 25, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 9, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 40.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 35.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 35.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 10, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 10, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 10, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 45.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 19, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 17, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 11, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 60.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 50.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 45.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 17, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 11, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 82, + "top_1_hit": false, + "top_1_minute_error": 73.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 72, + "top_1_hit": false, + "top_1_minute_error": 73.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 67, + "top_1_hit": false, + "top_1_minute_error": 73.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 62, + "top_1_hit": false, + "top_1_minute_error": 67.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 60, + "top_1_hit": false, + "top_1_minute_error": 67.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 61.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 55, + "top_1_hit": false, + "top_1_minute_error": 61.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 52, + "top_1_hit": false, + "top_1_minute_error": 60.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 49, + "top_1_hit": false, + "top_1_minute_error": 57.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 50.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 44, + "top_1_hit": false, + "top_1_minute_error": 50.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 42, + "top_1_hit": false, + "top_1_minute_error": 50.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 45.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 11, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 6.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 44.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 34.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 40.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 12, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 12, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 12, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 89.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 8, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 9, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 9, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 9, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 14, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 16, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 13, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 8, + "top_1_hit": false, + "top_1_minute_error": 21.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 4.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 22.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 26, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 13, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 37, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 42, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 13, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 50, + "top_1_hit": false, + "top_1_minute_error": 32.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 16.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 11.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 2.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 2.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 2.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 10.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 10.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 14, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 40, + "top_1_hit": false, + "top_1_minute_error": 2.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 39, + "top_1_hit": false, + "top_1_minute_error": 10.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 46, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 49, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 52, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 54, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 55, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 60, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 60, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 60, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 14, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 84, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 92, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 97, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 99, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 101, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 105, + "top_1_hit": false, + "top_1_minute_error": 37.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 14, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 109, + "top_1_hit": false, + "top_1_minute_error": 90.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 34.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 34.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 3.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 8, + "top_1_hit": false, + "top_1_minute_error": 3.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 11, + "top_1_hit": false, + "top_1_minute_error": 3.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 12.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 16, + "top_1_hit": false, + "top_1_minute_error": 15.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 19, + "top_1_hit": false, + "top_1_minute_error": 18.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 26, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 15, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 1, + "top_1_hit": true, + "top_1_minute_error": 0.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 11, + "top_1_hit": false, + "top_1_minute_error": 3.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 16, + "top_1_hit": false, + "top_1_minute_error": 15.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 26, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 34, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 39, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 46, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 61, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 15, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 46, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 53, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 56, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 58, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 61, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 64, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 66, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 69, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 71, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 76, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 81, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 15, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 91, + "top_1_hit": false, + "top_1_minute_error": 20.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 28 + }, + { + "case_ordinal": 16, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 30 + }, + { + "case_ordinal": 16, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 11.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 11.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 11.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 11.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 11.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 11.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 16, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 61, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": false, + "candidate_count": 61, + "delivery_width_minutes": 60 + }, + { + "case_ordinal": 16, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 60 + }, + { + "case_ordinal": 16, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 27.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 25.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 59, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 16, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 31.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 16, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 119, + "top_1_hit": false, + "top_1_minute_error": 30.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 35.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 19, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 8, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 5, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 43.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 17, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 25, + "top_1_hit": false, + "top_1_minute_error": 35.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 17, + "top_1_hit": false, + "top_1_minute_error": 27.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 15, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 12, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 10, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 5, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 17, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 51, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 48, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 47, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 46, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 41, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 36, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 30, + "top_1_hit": false, + "top_1_minute_error": 38.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 17, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 20, + "top_1_hit": false, + "top_1_minute_error": 29.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 10.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 12.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 18.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 18.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 18, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 14.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 18.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 22.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 57, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 61, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 18, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 104, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 114, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 18, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 117, + "top_1_hit": false, + "top_1_minute_error": 23.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 44.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 9.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 5.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 44.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 19, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 2, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 6, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 19, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 19, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 7, + "top_1_hit": false, + "top_1_minute_error": 1.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": -30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": -20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": -15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 22, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": -10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": -8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": -5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": -3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 18.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 0, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 21, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 3, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 5, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 16, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 8, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 13, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 10, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 11, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 15, + "radius_minutes": 15, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 8, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 20, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": 30, + "radius_minutes": 15, + "truth_in_window": false, + "truth_in_candidates": false, + "true_rank": null, + "top_1_hit": false, + "top_1_minute_error": 28.0, + "delivery_covers_truth": false, + "candidate_count": 31, + "delivery_width_minutes": 31 + }, + { + "case_ordinal": 20, + "offset_minutes": -30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 56.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": -20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 24, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": -15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": -10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": -8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": -5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": -3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 27, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 0, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 3, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 5, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 8, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 29, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 10, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 28, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 15, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 23, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 20, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 18, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": 30, + "radius_minutes": 30, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 8, + "top_1_hit": false, + "top_1_minute_error": 13.0, + "delivery_covers_truth": true, + "candidate_count": 61, + "delivery_width_minutes": 61 + }, + { + "case_ordinal": 20, + "offset_minutes": -30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 63, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": -20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 55, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": -15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 50, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": -10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 45, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": -8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 43, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": -5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 40, + "top_1_hit": false, + "top_1_minute_error": 65.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": -3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 38, + "top_1_hit": false, + "top_1_minute_error": 63.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 0, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 35, + "top_1_hit": false, + "top_1_minute_error": 56.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 3, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 56.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 5, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 32, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 8, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 10, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 15, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 20, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + }, + { + "case_ordinal": 20, + "offset_minutes": 30, + "radius_minutes": 60, + "truth_in_window": true, + "truth_in_candidates": true, + "true_rank": 31, + "top_1_hit": false, + "top_1_minute_error": 19.0, + "delivery_covers_truth": true, + "candidate_count": 121, + "delivery_width_minutes": 121 + } + ], + "summary": [ + { + "radius_minutes": 15, + "offset_minutes": -30, + "trial_count": 20, + "truth_in_window_rate": 0.0, + "top_1_rate": 0.0, + "delivery_coverage_rate": 0.0, + "mean_absolute_minute_error": 29.8 + }, + { + "radius_minutes": 15, + "offset_minutes": -20, + "trial_count": 20, + "truth_in_window_rate": 0.0, + "top_1_rate": 0.0, + "delivery_coverage_rate": 0.0, + "mean_absolute_minute_error": 22.75 + }, + { + "radius_minutes": 15, + "offset_minutes": -15, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.1, + "delivery_coverage_rate": 0.95, + "mean_absolute_minute_error": 15.35 + }, + { + "radius_minutes": 15, + "offset_minutes": -10, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 11.15 + }, + { + "radius_minutes": 15, + "offset_minutes": -8, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 8.05 + }, + { + "radius_minutes": 15, + "offset_minutes": -5, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 7.8 + }, + { + "radius_minutes": 15, + "offset_minutes": -3, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 8.3 + }, + { + "radius_minutes": 15, + "offset_minutes": 0, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 8.65 + }, + { + "radius_minutes": 15, + "offset_minutes": 3, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 8.0 + }, + { + "radius_minutes": 15, + "offset_minutes": 5, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 8.85 + }, + { + "radius_minutes": 15, + "offset_minutes": 8, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 10.75 + }, + { + "radius_minutes": 15, + "offset_minutes": 10, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 0.95, + "mean_absolute_minute_error": 11.05 + }, + { + "radius_minutes": 15, + "offset_minutes": 15, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.15, + "delivery_coverage_rate": 0.95, + "mean_absolute_minute_error": 12.3 + }, + { + "radius_minutes": 15, + "offset_minutes": 20, + "trial_count": 20, + "truth_in_window_rate": 0.0, + "top_1_rate": 0.0, + "delivery_coverage_rate": 0.0, + "mean_absolute_minute_error": 15.8 + }, + { + "radius_minutes": 15, + "offset_minutes": 30, + "trial_count": 20, + "truth_in_window_rate": 0.0, + "top_1_rate": 0.0, + "delivery_coverage_rate": 0.0, + "mean_absolute_minute_error": 30.9 + }, + { + "radius_minutes": 30, + "offset_minutes": -30, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.1, + "delivery_coverage_rate": 0.95, + "mean_absolute_minute_error": 23.4 + }, + { + "radius_minutes": 30, + "offset_minutes": -20, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 18.05 + }, + { + "radius_minutes": 30, + "offset_minutes": -15, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 17.05 + }, + { + "radius_minutes": 30, + "offset_minutes": -10, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 15.95 + }, + { + "radius_minutes": 30, + "offset_minutes": -8, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 17.1 + }, + { + "radius_minutes": 30, + "offset_minutes": -5, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 17.15 + }, + { + "radius_minutes": 30, + "offset_minutes": -3, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 16.7 + }, + { + "radius_minutes": 30, + "offset_minutes": 0, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 16.5 + }, + { + "radius_minutes": 30, + "offset_minutes": 3, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 16.25 + }, + { + "radius_minutes": 30, + "offset_minutes": 5, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 15.4 + }, + { + "radius_minutes": 30, + "offset_minutes": 8, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 14.8 + }, + { + "radius_minutes": 30, + "offset_minutes": 10, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 14.8 + }, + { + "radius_minutes": 30, + "offset_minutes": 15, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 14.35 + }, + { + "radius_minutes": 30, + "offset_minutes": 20, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 0.95, + "mean_absolute_minute_error": 16.65 + }, + { + "radius_minutes": 30, + "offset_minutes": 30, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.15, + "delivery_coverage_rate": 0.95, + "mean_absolute_minute_error": 19.1 + }, + { + "radius_minutes": 60, + "offset_minutes": -30, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 27.35 + }, + { + "radius_minutes": 60, + "offset_minutes": -20, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 28.6 + }, + { + "radius_minutes": 60, + "offset_minutes": -15, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 25.05 + }, + { + "radius_minutes": 60, + "offset_minutes": -10, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 24.75 + }, + { + "radius_minutes": 60, + "offset_minutes": -8, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 24.75 + }, + { + "radius_minutes": 60, + "offset_minutes": -5, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 24.45 + }, + { + "radius_minutes": 60, + "offset_minutes": -3, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 26.75 + }, + { + "radius_minutes": 60, + "offset_minutes": 0, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 26.5 + }, + { + "radius_minutes": 60, + "offset_minutes": 3, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 29.25 + }, + { + "radius_minutes": 60, + "offset_minutes": 5, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 27.35 + }, + { + "radius_minutes": 60, + "offset_minutes": 8, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 27.35 + }, + { + "radius_minutes": 60, + "offset_minutes": 10, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 27.35 + }, + { + "radius_minutes": 60, + "offset_minutes": 15, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 27.1 + }, + { + "radius_minutes": 60, + "offset_minutes": 20, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 25.95 + }, + { + "radius_minutes": 60, + "offset_minutes": 30, + "trial_count": 20, + "truth_in_window_rate": 1.0, + "top_1_rate": 0.05, + "delivery_coverage_rate": 1.0, + "mean_absolute_minute_error": 31.85 + } + ], + "widening_geometry": { + "scored_radii": [ + 15, + 30, + 60 + ], + "radius_120_scored": false, + "truth_in_window_condition": "abs(reported_offset_minutes) <= radius_minutes", + "radius_15_first_integer_minute_outside": 16, + "all_tested_offsets_within_radius_30_60_120": true, + "boundary": "Geometry rescues candidate inclusion only, not ranking or delivery coverage; no population frequency without a reported-offset distribution." + } +} diff --git a/docs/research/reported_offset_2026_09_20.md b/docs/research/reported_offset_2026_09_20.md new file mode 100644 index 00000000..5ea12451 --- /dev/null +++ b/docs/research/reported_offset_2026_09_20.md @@ -0,0 +1,64 @@ +# 申报偏差敏感性评测(2026-09-20) + +## 结论与边界 + +**默认半径 ±15 分钟时,申报误差绝对值超过 15 分钟,记录真值就不在候选窗内;按整数分钟,第 16 分钟开始出窗。** 这是搜索窗几何边界,不需要知道真实用户偏差分布。 + +本轮完整评测预先固定 15 个偏移 × 3 档半径 × 20 例,共 900 组合。修正版全部运行完成,validator 排除 0 例;实测结果以下列最终 JSON 为准,不能用中途旧版本初扫代替。 + +**区间覆盖不等于分钟命中,模拟偏差敏感性不等于真实用户准确率。** 不运行六题真值方向回放,不以用户认可作真值,不调参、不改生产打分、不打开确认门。 + +## 口径 + +| 项目 | 口径 | +| --- | --- | +| 数据集 | `references/real_case_calibration/minute_rectification_holdout_v3.json`,20 例公开 AA,每例 3 件事件,已曝光 | +| ayanamsa / node mode | `raman` / `mean` | +| 申报偏移 | 0、±3、±5、±8、±10、±15、±20、±30 分钟 | +| 搜索半径 | ±15 / ±30 / ±60 分钟;±120 仅说明几何边界,未评分 | +| 候选步长 | 1 分钟,含两端;不是生产 2 分钟抽样 | +| 12 文件历史身份 | `b15d9ea15227cd58`(完整值见 JSON) | +| 实际评分路径 | 原生 event contribution matrix;不是 T2 的 shadow fact ranker | +| 额外身份 | JSON 的 `production_scoring_sha256` 与 `research_implementation_sha256` 分别绑定扩展评分文件和研究适配文件;显式文件清单不是完整传递依赖锁 | +| 头名与排名 | 分数降序,再用既有 `_opaque_winner` 同源 SHA-256 规则排同分;不选择最接近真值的候选 | +| 交付区间 | 初始未答题:signature clusters 中落后头名不足 8 分的仍有效簇外包范围;无新回答、无淘汰,不是完整问答链交付效果 | +| 跨午夜 | 候选、误差及区间使用完整日期;离线矩阵按候选日期分组,合并后全窗排序/聚类 | +| 独立性 | `is_blind_evaluation=false`,已曝光集;评分/区间定稿后才揭示标签,仅证明程序标签隔离 | + +## 实测表 + +每格依次为 **真值在窗比例 / 哈希头名命中 / 初始交付区间覆盖**,分母均为 20 例。列为申报偏移分钟,行为搜索半径。统一口径:v3、raman/mean、1 分钟步长;历史打分身份 `b15d9ea15227cd58`,扩展原生评分身份 `115c3fbcffdaff49`,研究适配身份 `8d0c613dc0900adb`。 + +| 半径 | -30 | -20 | -15 | -10 | -8 | -5 | -3 | 0 | +3 | +5 | +8 | +10 | +15 | +20 | +30 | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| ±15 | 0/0/0 | 0/0/0 | 100/10/95 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/95 | 100/15/95 | 0/0/0 | 0/0/0 | +| ±30 | 100/10/95 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/95 | 100/15/95 | +| ±60 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | 100/5/100 | + +表内全部数字单位为 %。这只是 20 例公开已曝光案例的条件敏感性,不附会总体置信度。即使真值在候选窗中,±15 的 -15/+10/+15 档及 ±30 的 -30/+20/+30 档仍各有一例未被初始交付范围覆盖;窗口包含与交付覆盖必须分开。±60 的所有采样偏移虽保持覆盖,但真正选中的头名只有 5%,不能用宽区间覆盖声称分钟准确。 + +修正版与错误初版的汇总比例恰好相同,不表示初版评分正确;身份、逐候选计算与跨日回归以修正版为准。 + +## 放宽能救回什么 + +- 真值在候选窗的条件始终是 `abs(offset) <= radius`。本次偏移最大绝对值 30,因此半径 30 / 60 / 120 在几何上都能包含本次所有偏移标签。 +- 对 ±15 已出窗的 ±20 / ±30 两组偏移,放宽到 ±30 可把**候选窗包含率**从 0 恢复到 100%;±60 / ±120 对这些特定偏移不再增加几何包含率。 +- 半径 ±30 自第 31 个整数分钟偏移出窗,±60 自第 61 个、±120 自第 121 个。放宽不是无条件保证,更不代表排名命中或交付覆盖同样恢复。 +- ±120 未跑评分,不报告其头名命中率或交付覆盖率。真实用户申报偏差分布未知,不估算现实用户中有多少人能被放宽救回。 + +## 初扫作废与独立复核 + +首版正确枚举了跨日候选,但矩阵内 transition proximity 仍共用窗口起日,独立审查发现跨日候选得分偏移。因此旧初扫作废,修正版标记 `replay_revision=candidate_date_grouped_v2`,并登记 `supersedes=initial_sweep_invalidated_cross_midnight_transition_anchor`。 + +修复仅在本次新脚本 `score_window()` 做按日期分组适配,未改冻结打分文件。实引擎回归验证跨日窗口的分组计算与逐候选独立重算相同;聚类和交付范围在合并后全窗运行。**生产日期处理未在本单修复,不能称为生产端到端回放。** + +## 可复算产物与验收 + +```bash +python scripts/research/reported_offset_sweep.py --json +python -m pytest tests/test_reported_offset_research.py tests/test_rectification_validation_integrity_gate.py -q +``` + +`reported_offset_2026_09_20.json` 保留逐例四项必要输出:真值是否在窗、真值排名、头名分钟误差、交付区间覆盖,以及每个格子的分母、全部参数和实现哈希。仅保存公开集序号,不保存出生分钟、坐标或事件正文。 + +排名不能与 T2 的竞争排名 top-1/top-3 混比:T2 按旧协议允许并列多个分钟同时算第一;本表把同分完全按哈希打破。最终测试数字与已知基线失败见 [进度](../tasks/PROGRESS-rectification-validation-20260920.md)。真实分布采集与新未曝光样本见 [v5 协议](sealed_holdout_v5_protocol_2026_09_20.md)。 diff --git a/docs/research/sealed_holdout_rerun_2026_09_20.freeze.json b/docs/research/sealed_holdout_rerun_2026_09_20.freeze.json new file mode 100644 index 00000000..62ad97bd --- /dev/null +++ b/docs/research/sealed_holdout_rerun_2026_09_20.freeze.json @@ -0,0 +1,43 @@ +{ + "record_version": "exposed-v3-fixed-protocol-rerun-v1", + "frozen_at_utc": "2026-09-20T03:27:26.670869+00:00", + "dataset_path": "references/real_case_calibration/minute_rectification_holdout_v3.json", + "dataset_sha256": "45aa76ab79dfc6c6404b6252083e1aa5c7e2e2b7de811fc11042a8ad2da3f0ea", + "algorithm_version": "birth-time-event-fact-ranker-v4-shadow", + "implementation_sha256": "b15d9ea15227cd58b3097555537fa7c37fb5a993905ada4f5aa04f52de1f8f18", + "files": [ + "scripts/active_rectification_event_engine.py", + "scripts/active_rectification_events.py", + "scripts/ashtakavarga.py", + "scripts/dasha_analyzer.py", + "scripts/divisional_charts_extended.py", + "scripts/domain_calculation_service.py", + "scripts/jaimini.py", + "scripts/minute_rectification_fact_ranker_v4.py", + "scripts/minute_rectification_feature_facts_v4.py", + "scripts/narayana_dasha.py", + "scripts/shadbala.py", + "scripts/varga.py" + ], + "historical_frozen_sha256": "f41c298dd6cdcebe7a93e632f7191954be7987f6012ca2a34cecb6e447fbf196", + "evaluator_sha256": "3bf35de5f8a0b9588808178d47ba2457c398cf2fd2d2fcb03c5c71420f998da5", + "ayanamsa": "raman", + "node_mode": "mean", + "candidate_radius_minutes": [ + 10 + ], + "minute_step": 1, + "release_metrics": { + "top_1_rate_minimum": 0.6, + "top_3_rate_minimum": 0.85, + "mean_absolute_minute_error_maximum": 2.0, + "false_confirmation_rate_maximum": 0.05, + "correct_insufficient_evidence_rejection_rate_minimum": 0.9 + }, + "results_previously_seen": true, + "official_valid_independent_blind": false, + "must_not_use_for_tuning": true, + "tie_breaker": "sha256(benchmark_id:case_id:candidate_time); published minute is never passed to the ranker", + "metric_rank_definition": "competition_rank_1_plus_strictly_higher_scores_legacy_protocol", + "extra_metric_rank_definition": "score_desc_then_existing_opaque_sha256_total_order" +} diff --git a/docs/research/sealed_holdout_rerun_2026_09_20.json b/docs/research/sealed_holdout_rerun_2026_09_20.json new file mode 100644 index 00000000..46f688e0 --- /dev/null +++ b/docs/research/sealed_holdout_rerun_2026_09_20.json @@ -0,0 +1,626 @@ +{ + "scope": "fixed_protocol_previously_exposed_v3_rerun", + "evaluated_on": "2026-09-20", + "frozen_record": { + "record_version": "exposed-v3-fixed-protocol-rerun-v1", + "frozen_at_utc": "2026-09-20T03:27:26.670869+00:00", + "dataset_path": "references/real_case_calibration/minute_rectification_holdout_v3.json", + "dataset_sha256": "45aa76ab79dfc6c6404b6252083e1aa5c7e2e2b7de811fc11042a8ad2da3f0ea", + "algorithm_version": "birth-time-event-fact-ranker-v4-shadow", + "implementation_sha256": "b15d9ea15227cd58b3097555537fa7c37fb5a993905ada4f5aa04f52de1f8f18", + "files": [ + "scripts/active_rectification_event_engine.py", + "scripts/active_rectification_events.py", + "scripts/ashtakavarga.py", + "scripts/dasha_analyzer.py", + "scripts/divisional_charts_extended.py", + "scripts/domain_calculation_service.py", + "scripts/jaimini.py", + "scripts/minute_rectification_fact_ranker_v4.py", + "scripts/minute_rectification_feature_facts_v4.py", + "scripts/narayana_dasha.py", + "scripts/shadbala.py", + "scripts/varga.py" + ], + "historical_frozen_sha256": "f41c298dd6cdcebe7a93e632f7191954be7987f6012ca2a34cecb6e447fbf196", + "evaluator_sha256": "3bf35de5f8a0b9588808178d47ba2457c398cf2fd2d2fcb03c5c71420f998da5", + "ayanamsa": "raman", + "node_mode": "mean", + "candidate_radius_minutes": [ + 10 + ], + "minute_step": 1, + "release_metrics": { + "top_1_rate_minimum": 0.6, + "top_3_rate_minimum": 0.85, + "mean_absolute_minute_error_maximum": 2.0, + "false_confirmation_rate_maximum": 0.05, + "correct_insufficient_evidence_rejection_rate_minimum": 0.9 + }, + "results_previously_seen": true, + "official_valid_independent_blind": false, + "must_not_use_for_tuning": true, + "tie_breaker": "sha256(benchmark_id:case_id:candidate_time); published minute is never passed to the ranker", + "metric_rank_definition": "competition_rank_1_plus_strictly_higher_scores_legacy_protocol", + "extra_metric_rank_definition": "score_desc_then_existing_opaque_sha256_total_order" + }, + "implementation_hash_matches_at_replay": true, + "dataset_hash_matches_at_replay": true, + "source_audit_status": "corrected_known_date_errors", + "validation_status": "blocked_awaiting_public_aa_cases", + "valid_public_aa_cases": 20, + "excluded_cases": [], + "trial_count": 20, + "trials": [ + { + "case_ordinal": 1, + "candidate_count": 21, + "event_count": 3, + "true_rank": 5, + "opaque_true_rank": 8, + "minute_error": 10, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 2, + "candidate_count": 21, + "event_count": 3, + "true_rank": 2, + "opaque_true_rank": 8, + "minute_error": 3, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "insufficient_events", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 3, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 12, + "minute_error": 5, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 4, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 1, + "minute_error": 0, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "insufficient_events", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 5, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 10, + "minute_error": 1, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 6, + "candidate_count": 21, + "event_count": 3, + "true_rank": 7, + "opaque_true_rank": 12, + "minute_error": 6, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 7, + "candidate_count": 21, + "event_count": 3, + "true_rank": 5, + "opaque_true_rank": 7, + "minute_error": 9, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 8, + "candidate_count": 21, + "event_count": 3, + "true_rank": 5, + "opaque_true_rank": 8, + "minute_error": 10, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "insufficient_events", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 9, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 3, + "minute_error": 1, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "missing_mandatory_layers", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "missing_mandatory_layers", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 10, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 8, + "minute_error": 5, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 11, + "candidate_count": 21, + "event_count": 3, + "true_rank": 6, + "opaque_true_rank": 17, + "minute_error": 5, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 12, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 4, + "minute_error": 5, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 13, + "candidate_count": 21, + "event_count": 3, + "true_rank": 5, + "opaque_true_rank": 7, + "minute_error": 10, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 14, + "candidate_count": 21, + "event_count": 3, + "true_rank": 17, + "opaque_true_rank": 19, + "minute_error": 10, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 15, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 7, + "minute_error": 9, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "missing_mandatory_layers", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "missing_mandatory_layers", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 16, + "candidate_count": 21, + "event_count": 3, + "true_rank": 12, + "opaque_true_rank": 12, + "minute_error": 10, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 17, + "candidate_count": 21, + "event_count": 3, + "true_rank": 6, + "opaque_true_rank": 6, + "minute_error": 6, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 18, + "candidate_count": 21, + "event_count": 3, + "true_rank": 11, + "opaque_true_rank": 14, + "minute_error": 10, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "insufficient_events", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 19, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 6, + "minute_error": 4, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + }, + { + "case_ordinal": 20, + "candidate_count": 21, + "event_count": 3, + "true_rank": 1, + "opaque_true_rank": 7, + "minute_error": 10, + "would_confirm": false, + "false_confirmation": false, + "insufficient_evidence_rejected": true, + "full_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ], + "sparse_trial_reasons": [ + "unique_minute_not_found", + "insufficient_events", + "insufficient_domains", + "neighbor_stability_not_passed", + "leave_one_event_out_not_passed", + "top_candidate_feature_not_unique", + "fact_ranker_v4_holdout_not_ready" + ] + } + ], + "metrics": { + "top_1_rate": 0.45, + "top_3_rate": 0.5, + "mean_absolute_minute_error": 6.45, + "false_confirmation_rate": 0.0, + "correct_insufficient_evidence_rejection_rate": 1.0, + "confirmation_coverage_rate": 0.0 + }, + "metric_gates_passed": false, + "opaque_exact_top_1_rate": 0.05, + "opaque_exact_top_3_rate": 0.1, + "official_valid_independent_blind": false, + "official_blind_trial_count": 0, + "is_blind_evaluation": false, + "truth_hidden_from_ranker": true, + "results_previously_seen": true, + "verified_minute_claim_allowed": false, + "status": "blocked_independent_blind_evidence", + "boundary": "Three-event low-information protocol, not a mathematical accuracy lower bound and not representative of real sessions. Historical v3/v4 exposure cannot be undone by freezing today's scorer. No tuning or release claims." +} diff --git a/docs/research/sealed_holdout_rerun_2026_09_20.md b/docs/research/sealed_holdout_rerun_2026_09_20.md new file mode 100644 index 00000000..0a725440 --- /dev/null +++ b/docs/research/sealed_holdout_rerun_2026_09_20.md @@ -0,0 +1,72 @@ +# 当前冻结实现的 v3 固定口径重跑(2026-09-20) + +## 结论 + +**已补上当前实现可归属、可复算的成绩;没有补成新的独立官方盲测。** 既有 v3/v4 人物与成绩已经曝光,重新冻结今天的打分文件不能消除曝光。任务书 T2 中「首次口径干净的官方盲测」在既有 BUG-427 / BUG-428 红线下仍为 **blocked**,不得改名通过。确认门保持关闭。 + +这是每例只有 3 件事的低信息量(任务书称「下限」)口径,不代表真实会话;事件少不等于数学上保证成绩更低,因此也不是准确率下界。 + +## 固定口径与可审计文件 + +| 项目 | 本次实际值 | +| --- | --- | +| 数据集 | `references/real_case_calibration/minute_rectification_holdout_v3.json` | +| 数据集 SHA-256 | `45aa76ab79dfc6c6404b6252083e1aa5c7e2e2b7de811fc11042a8ad2da3f0ea` | +| 计算规格 | ayanamsa `raman` / node mode `mean` | +| 半径 / 步长 | ±10 分钟 / 1 分钟,含两端;真值为窗心 | +| 算法 | `birth-time-event-fact-ranker-v4-shadow`,不是生产 V9 问答链 | +| 当前打分身份 | `b15d9ea15227cd58b3097555537fa7c37fb5a993905ada4f5aa04f52de1f8f18`(前缀 `b15d9ea15227cd58`) | +| 样本 / 事件 | 20 例 / 每例 3 件;validator 排除 0 例 | +| 资料审计状态 | `corrected_known_date_errors`,不等同全新独立人工重审 | +| 排名标签隔离 | 排序和稀疏证据判定完成后才读取真值标签 | +| 独立官方盲测 | `official_valid_independent_blind=false`;有效官方盲测试次仍为 0 | + +冻结记录:`sealed_holdout_rerun_2026_09_20.freeze.json`。先以独占创建模式记录 12 个文件、数据集与评测器字节哈希,再执行评分;评分前验证全部身份,任何漂移即拒绝。历史 v3 `frozen_scoring` 原值 `f41c298dd6cdcebe…` 未改。 + +脱敏逐例结果:`sealed_holdout_rerun_2026_09_20.json`,只含公开集序号、排名、误差、门禁原因,不含出生分钟、坐标或事件正文。复算: + +```bash +python scripts/research/sealed_holdout_rerun.py --json +``` + +## 五项预注册指标与门槛 + +本表统一口径:v3 / `raman` / `mean` / ±10 / 1 分钟 / `b15d9ea15227cd58`。 + +| 指标 | 实测 | 预注册门槛 | 数值对照 | +| --- | ---: | ---: | --- | +| top-1(旧协议竞争排名,含并列) | 0.45 | ≥0.60 | 未通过 | +| top-3(旧协议竞争排名,含并列) | 0.50 | ≥0.85 | 未通过 | +| 头名平均绝对分钟误差(哈希打破并列) | 6.45 分钟 | ≤2.00 分钟 | 未通过 | +| 误确认率(全样本分母) | 0.00 | ≤0.05 | 通过,但无确认覆盖,不能证明确认精度 | +| 信息不足正确拒绝率(每例仅首件事件) | 1.00 | ≥0.90 | 通过 | +| 确认覆盖率(补充) | 0.00 | 本单禁止抬高 | 保持关闭 | +| 作为独立盲测发布成绩的资格 | 不适用 | 新封存、未曝光、独立审计 | blocked,不可当发布指标 | + +**重要口径纠正:旧 top-1/top-3 不是哈希选中那个分钟的命中率。** 现有评测的 `true_rank = 1 + 严格高于真值分数的候选数` 会把同分的多个分钟全算第一;本次为保证与旧协议可比保留它,同时增加哈希全序的真实选中指标,不暗改预注册定义。 + +| 附加指标(同一上述口径) | 实测 | +| --- | ---: | +| `_opaque_winner` 真正选中的头名命中 | 0.05 | +| 同一哈希全序中的 top-3 命中 | 0.10 | + +这两个数字不能与旧 0.45/0.50 混称同一种准确率。没有据此改权重、换半径、换案例或剔除失败例。 + +## 契约改了什么、没有改什么 + +- `current_tree_scorer` 绑定本次真实哈希及新报告,新增固定口径试次数与成绩;旧 `current_tree_unfrozen_diagnostic` 历史块保留。 +- `source_audit_status` 对齐数据集;`evaluated_on` 表示最新固定口径重跑日期,并新增作用域说明。历史顶层指标仍有 `metrics_produced_by` 的旧日期与来源,未伪装成本次结果。 +- `status`、`valid_public_aa_cases`、`required_cases`、`top_1_rate`、`confirmation_coverage_rate`、`sealed_benchmark_id` 六个 runtime 键的值与类型全不变;尤其 `status=not_ready`、确认覆盖率为零。 +- `official_eval_implementation_hash_matches=false`、`official_eval_trial_count=0` 保留;新增重跑记录不冒充已重新获得独立性。 +- 未改任何冻结打分文件、生产判定、现有 `_candidate_moments()` 或 `request_from_case()`。 + +## 防复发与既有断言变更 + +新增 `tests/test_sealed_holdout_contract_freshness.py` 钉死当前树身份、审计状态、冻结记录、报告汇总与独立性声明;冻结身份不符必须在计算前拒绝。`tests/test_rectification_validation_integrity_gate.py` 将其和偏差评测回归纳入现有 `test_rectification_*.py` 快速门,不改 workflow。 + +| 原断言 | 新断言 | 原因 | +| --- | --- | --- | +| Python / 前端:`current_tree_scorer.implementation_sha256` 等于旧 post-audit sidecar 的哈希 | 等于新固定口径报告 `frozen_record.implementation_sha256`,另验证当前树与新报告 | 契约当前树块的含义就是当前实现;继续绑定旧报告会强制过期。新增身份断言更严格,不弱化门禁 | +| 官方标志 false、官方试次 0、六键等于历史报告、确认门全套行为断言 | 原值保留,另加固定口径重跑身份/试次和独立盲测 false | 固定口径重跑不是新的独立盲测,不借元数据迁移打开确认门 | + +本机 Python 3.11.7 可跑;`python3` Windows 别名退出 49。该任务定向命令的最终测试清单与基线失败对照由 `PROGRESS-rectification-validation-20260920.md` 统一记录。特别是既有 v2 冻结身份测试不属本次刷新范围,不得为了通过而改其历史封存或断言。 diff --git a/docs/research/sealed_holdout_v5_protocol_2026_09_20.md b/docs/research/sealed_holdout_v5_protocol_2026_09_20.md new file mode 100644 index 00000000..476adefd --- /dev/null +++ b/docs/research/sealed_holdout_v5_protocol_2026_09_20.md @@ -0,0 +1,132 @@ +# 封存集 v5 采集协议(2026-09-20) + +> 状态:**协议已拟定,未采集事件、未建成数据集、未评分、未完成独立人工审阅**。本文件只交付 T3,不等于 BUG-980 已解决。 +> 任务:[TASK-rectification-validation-integrity-20260920](../tasks/TASK-rectification-validation-integrity-20260920.md)。关联 BUG-098 / BUG-428 / BUG-980。 +> 本文所有样本量、事件数、半径与工时均为**协议目标或估算**,不是模型成绩;本轮不产生评分实现哈希或准确率。未来评分必须附完整实现哈希、数据集哈希、ayanamsa、node mode、半径、步长和试验登记号。 + +## 1. 要证明什么,以及明确不证明什么 + +v5 的目标是同时满足 **封存 + 每例至少 7 件带年月事件 + 至少 4 个领域**:事件密度对齐 v4,隔离纪律至少达到 v3 设计目标,并修补“程序不读真值”不等于“人未看过成绩”的缺口。拟获得至少 25 个合格人物;下列 26 人只是备选池,不保证其事件覆盖已达标,不足时须在评分前按同一标准扩池。 + +**在 v5 就绪之前,任何对外的分钟级准确率说法都没有依据。** v5 就绪也仅是允许开始验证,不是默认验证成功;还必须完成预注册评测、独立审阅与发布决策。 + +- T2 的数字是 v3 每例仅 3 件事的**低事件量口径**,不是对真实会话准确率的保证,更不是数学意义上的性能下界。重新冻结当前实现可以恢复“成绩归属于哪个实现”的可追溯性,不能消除旧人物、旧事件或旧成绩的曝光。 +- T1 的数字是“申报偏差为 X 时会怎样”的**敏感性**,不是实际用户准确率;未知真实用户的申报偏差分布时,不得把各偏移档等权平均后宣传为产品表现。 +- 真值藏在排名之后只证明程序的标签隔离;独立盲测还要求未用其人物/事件/结果调参。BUG-428 已规定:**结果已被看过的案例不得再计入盲测**,自动化迁移的审阅记录也不等于新的传记级人审。 +- Rodden-AA 表示出生来源等级,不保证记录无舍入、无转录差错,更不保证真实出生瞬间精确到某分钟。评测标签称为“有来源的记录时间”,不能抹平记录误差。 +- 本协议不改打分、阈值、产品行为或确认门。`accepted` 不等于 `confirmed`;即使未来达到指标,确认门是否开放仍须产品另行授权。 + +## 2. 候选池与来源筛查 + +### 2.1 本次核对的边界 + +2026-09-20 通过公开网络搜索,逐人核对 Astro-Databank 或其同站生日索引返回的 **Rodden Rating = AA** 与出生资料来源类别。本机对部分原文页直接读取返回校验页,因此以下状态仅为**可追溯的候选筛查**,不是出生证原件已取得、源页已归档或独立人工审阅已完成。正式入集前必须重新打开逐人原文、复核 Source Notes 并登记页面修订号/检索时间;若无法核实或等级降级,按资料不合格替换,不得把搜索摘要升级为完成证据。 + +只列人物和资料来源类型,不采集、复制或展示出生分钟、坐标和生平事件内容。`BC/BR in hand` / `Quoted BC/BR` 是来源网站的标签,不是声称本项目持有原件。事件来源目前只是允许的来源类型,尚未检索或标注。 + +| # | 公开人物 | AA 来源页与出生资料类型(候选筛查) | 后续事件来源类型(待采集) | +| --- | --- | --- | --- | +| 1 | Albert Einstein | [Astro-Databank](https://www.astro.com/astro-databank/Einstein,_Albert);BC/BR in hand | 诺贝尔官方、大学档案、公开传记 | +| 2 | Marie Curie | [Astro-Databank](https://www.astro.com/astro-databank/Curie,_Marie);Quoted BC/BR,Source Notes 指向洗礼记录 | 诺贝尔官方、研究机构档案、公开传记 | +| 3 | Emmanuel Macron | [Astro-Databank](https://www.astro.com/astro-databank/Macron,_Emmanuel);Quoted BC/BR | 政府公开档案、可靠新闻、公开传记 | +| 4 | Marilyn Monroe | [Astro-Databank](https://www.astro.com/astro-databank/Monroe,_Marilyn);BC/BR in hand | 电影机构档案、可靠新闻、公开传记 | +| 5 | Emma Watson | [Astro-Databank](https://www.astro.com/astro-databank/Watson,_Emma);Quoted BC/BR | 联合国公开档案、院校/电影机构记录、公开传记 | +| 6 | George Clooney | [Astro.com 同站 AA 索引](https://www.astro.com/adbvip/adbvip_05_06.htm);BC/BR in hand | 电影学院/电影节官方、可靠新闻、公开传记 | +| 7 | Tom Hanks | [Astro-Databank](https://www.astro.com/astro-databank/Hanks,_Tom);BC/BR in hand | 电影学院官方、可靠新闻、公开传记 | +| 8 | Meryl Streep | [Astro.com 同站 AA 索引](https://www.astro.com/adbvip/adbvip_06_22.htm);BC/BR in hand | 电影学院官方、院校档案、公开传记 | +| 9 | Johnny Depp | [Astro-Databank](https://www.astro.com/astro-databank/Depp,_Johnny);BC/BR in hand | 电影机构档案、可核实公开记录、可靠新闻 | +| 10 | Leonardo DiCaprio | [Astro-Databank](https://www.astro.com/astro-databank/DiCaprio,_Leonardo);BC/BR in hand | 电影学院官方、联合国公开档案、公开传记 | +| 11 | Albert Camus | [Astro-Databank](https://www.astro.com/astro-databank/Camus,_Albert);Quoted BC/BR | 诺贝尔官方、出版档案、公开传记 | +| 12 | Jean-Paul Sartre | [Astro-Databank](https://www.astro.com/astro-databank/Sartre,_Jean-Paul);BC/BR in hand | 诺贝尔官方、院校/出版档案、公开传记 | +| 13 | Simone de Beauvoir | [Astro-Databank](https://www.astro.com/astro-databank/Beauvoir,_Simone_de);BC/BR in hand | 文学奖/出版档案、院校档案、公开传记 | +| 14 | Victor Hugo | [Astro-Databank](https://www.astro.com/astro-databank/Hugo,_Victor);BC/BR in hand | 国家图书馆/议会档案、公开传记 | +| 15 | Marcel Proust | [Astro-Databank](https://www.astro.com/astro-databank/Proust,_Marcel);BC/BR in hand | 文学奖/出版档案、公开传记 | +| 16 | Marion Cotillard | [Astro-Databank](https://www.astro.com/astro-databank/Cotillard,_Marion);Quoted BC/BR | 电影学院/电影节官方、可靠新闻、公开传记 | +| 17 | Juliette Binoche | [Astro-Databank](https://www.astro.com/astro-databank/Binoche,_Juliette);Quoted BC/BR | 电影学院/电影节官方、可靠新闻、公开传记 | +| 18 | Catherine Deneuve | [Astro-Databank](https://www.astro.com/astro-databank/Deneuve,_Catherine);Quoted BC/BR | 电影机构档案、可靠新闻、公开传记 | +| 19 | Alain Delon | [Astro-Databank](https://www.astro.com/astro-databank/Delon,_Alain);Quoted BC/BR | 电影机构档案、可靠新闻、公开传记 | +| 20 | Sophie Marceau | [Astro-Databank](https://www.astro.com/astro-databank/Marceau,_Sophie);Quoted BC/BR | 电影机构档案、可靠新闻、公开传记 | +| 21 | Jean Dujardin | [Astro-Databank](https://www.astro.com/astro-databank/Dujardin,_Jean);BC/BR in hand | 电影学院/电影节官方、可靠新闻、公开传记 | +| 22 | Zinedine Zidane | [Astro-Databank](https://www.astro.com/astro-databank/Zidane,_Zinedine);BC/BR in hand | FIFA/俱乐部官方、可靠新闻、公开传记 | +| 23 | Kylian Mbappé | [Astro-Databank](https://www.astro.com/astro-databank/Mbapp%C3%A9,_Kylian);Quoted BC/BR | FIFA/俱乐部官方、可靠新闻、公开传记 | +| 24 | Antoine Griezmann | [Astro-Databank](https://www.astro.com/astro-databank/Griezmann,_Antoine);Quoted BC/BR | FIFA/俱乐部官方、可靠新闻、公开传记 | +| 25 | Serena Williams | [Astro.com 同站 AA 索引](https://www.astro.com/adbvip/adbvip_09_26.htm);Quoted BC/BR | WTA/赛事官方、可靠新闻、公开传记 | +| 26 | Muhammad Ali | [Astro-Databank](https://www.astro.com/astro-databank/Ali,_Muhammad);BC/BR in hand | 体育机构档案、可靠新闻、公开传记 | + +来源类别释义:[Rodden Rating](https://www.astro.com/astro-databank/Help:RR)、[Data Source](https://www.astro.com/astro-databank/Help:DataSource)。备选筛查中非 AA、无法明确核实 AA 的人物不进入此表;不能用关联人物条目的 AA 为目标人物背书。 + +### 2.2 与既有材料去重 + +基线 `5049e784` 中 v3 / v4 的人物集合相同,均为以下 20 人:Barack Obama、Angelina Jolie、Tiger Woods、Elizabeth Taylor、Elizabeth Montgomery、Pablo Picasso、Sigmund Freud、Salma Hayek、Steve Reich、Albert Brooks、Paul Ryan、Tom Kennedy、Andrew Windsor、Sean Lennon、Joseph Kennedy III、Kurt Cobain、John Robbins、Chad Everett、Bernd Eichinger、Iwao Takamoto。 + +本表 26 名候选与上述集合的**人物交集为 0**;不是只比较文件行号或 case_id。正式采集还须按姓名别名、出生来源页归一化身份,与仓库内其他已评分语料、v4 intake、已公开研究报告继续比对:**与 v3/v4 不重叠只是必要条件,不是无曝光证明**。姓名公开也不代表其事件内容可以用于调参。 + +此备选池存在明显的地区、职业、公共传记可得性偏倚。不能为凑四个领域把多次职业获奖改标成财富、迁居或家庭事件;不能把静态家庭属性塞进事件打分。每人未达到门槛时记为筛查未通过,评分前依协议扩池;不得看成绩后换人。 + +## 3. 标注协议:沿用 v4 四条,补上真正的封存 + +沿用 [holdout_v4_build_2026_09_14.md](holdout_v4_build_2026_09_14.md) 的四条约束,但**不继承 v4 的公开评价属性或人物**: + +1. **AA 记录**:出生数据只取已复核的公开 Astro-Databank AA 记录及 Source Notes。这里的“沿用”指原样保留来源内容和精度,不是从旧集拷贝人物。来源存在冲突、舍入或日期/时区解释问题时必须留标记;无法裁决不得强选有利于打分的一项。 +2. **独立事件来源**:每人至少 7 件可核实到年月的事件、覆盖至少 4 个现有可评分领域;事件来源可为 Wikipedia、Britannica、诺贝尔/电影学院/体育机构官方、政府/大学档案等。每件至少一处可核实公开来源,且 `independent_of_birth_source=true` 只能在事件来源实际独立于出生源后置真,不能只看 URL 不同。占星网站转述和依赖出生源的循环引用不算独立事件证据。 +3. **工作日期必须核实**:工作日期用经核实的公历,精度忠实于来源。只有年份的事件不能虚填月份来凑本协议的年月门槛;冲突必须由独立复核者裁决并记录,不沿用已被否定的错年。保存最小必要的出处、检索时间、精度、领域、审阅记录,遵守转载许可,不能大段复制受版权保护原文。 +4. **三档半径写在数据集上**:保留对照档 ±10 / ±30 / ±60,不以成绩决定跑哪些档;不把“真值所在簇”预写入可评分输入。正式采集任务还须在揭晓前冻结产品申报中心档 ±15 / ±30 / ±60 / ±120、偏移列表及跨午夜规则,作为独立的申报偏差敏感性附表,不混进真值居中成绩。 +5. **封存**:**v5 建成后,在产出第一份成绩之前不得被任何人读取事件内容做调参**;第一份成绩之后同样不得用 v5 调参。采集者/审核者因工作需要可读取来源,但不能参与基于该材料的打分选择;评分/调参团队只能看到协议、接口形状与另立的开发集。任何越界曝光必须登记,相关人物降为开放诊断材料,不能继续充当未见盲测。 + +## 4. 角色隔离、封存物与一次揭晓 + +| 阶段 | 执行角色与可见内容 | 必须保留的证据 | +| --- | --- | --- | +| 名单筛查 | 数据采集角色;仅来源资格与既有曝光比对 | 候选/排除清单、资料理由;不得用评分优劣筛选 | +| 事件标注 | 数据采集角色;出生来源与公开事件 | 每件来源、精度、领域、原始/裁决日期关联;不运行评分 | +| 独立人审 | 不参与调参的另一名人工审阅者 | 真实 `fresh_biography_audit` 记录;自动化不可自置 `independent_human_reviewed=true` | +| 冻结 | 评测保管角色 | 固定成员清单、持久分区、数据/事件/标签 SHA-256、来源审阅清单、冻结时间;访问权限与曝光日志 | +| 评分冻结 | 评分负责人;仅开发集与协议 | 精确 commit、实现文件清单/逐文件及聚合哈希、参数、依赖/引擎版本、并列规则、指标计算和拒绝定义 | +| 单次完整运行 | 隔离评测运行器 | ranker 输入中无真值标签;所有候选完成排名并持久化结果后才能关联标签;运行日志、退出码与输出哈希 | +| 揭晓与审阅 | 评测保管角色 + 独立审核者 | 首次全量报告、全部失败/无效项、阈值对照、执行环境与重试台账;不得只展示最佳档 | + +建议将未揭晓事件/标签包保存在独立受控存储,不提交到评分开发者可随手读取的工作树;仓库先只保存协议、匿名 manifest 与哈希。谁能读事件、谁能改评分必须分别登记。名单已公开、预训练语料可能包含名人生平等剩余污染风险必须写进报告,不能把访问隔离说成绝对无污染。 + +- 原始标签与候选排名分离;不得在并列时按接近真值挑选。沿用已冻结的 `_opaque_winner` 式不依赖真值的哈希打破规则。 +- 引擎口径拟固定 `ayanamsa=raman`、`node_mode=mean`;精确分钟排名使用 1 分钟候选步长。若需要与旧 2 分钟研究比较,另列预注册附表,不能合并分母或借用旧成绩。 +- 同一人全部事件只属于一个固定分区,不能通过换 case_id、加事件、修改旧标注来洗成新盲测。 +- 在首次运行前原样登记五项发布阈值和计算定义(top-1、top-3、MAE、误确认率、信息不足正确拒绝率),同时报告区间覆盖、确认覆盖率、真值在候选集中的比例与分母。信息不足子集及拒绝对照必须在揭晓前指定,不能从失败案例反向挑选。 +- `validator` 不通过的人物在评分前排除且列明理由。运行错误不应静默剔除;若是基础设施失败,保留原失败记录并只允许同一冻结实现/数据重跑。若源数据揭晓后发现错误,旧报告标无效/受影响,不删除历史,更不得把修订后的已见案例重新标成从未曝光。 +- BUG-098 要求的稳定分区与校准证据在此具体化为冻结 manifest、访问/曝光日志和独立验收;尚未有这些产物时,不得声称 v5 已建成。现存人审校验规则不能代替这些实际产物。 + +## 5. 工作量估算与完成门 + +以下是人工工时估算,不是已经投入的工时。按最终 25 人、每人至少 7 件事件,即至少 **175 件**、每件至少一处可核实公开来源计算;网络读取、资料不足与来源冲突可能使工作量上浮。 + +| 工作 | 估算依据 | 人工小时 | +| --- | --- | ---: | +| AA 原文复核、别名/历史曝光核对及补候选 | 25 人及少量替补的筛查 | 12–24 | +| 事件检索与首轮标注 | 175 件 × 15–30 分钟 | 44–88 | +| 独立复核与日期/领域冲突裁决 | 175 件 × 约 7.5–15 分钟 | 22–44 | +| 封存、权限分离、manifest、试运行接口校验和交接 | 不接触评分结果的工程与审阅 | 8–16 | +| **合计** | 取整;是人时,不是单人日历工期 | **约 86–172** | + +该区间不含新算法开发、不含真实用户招募,也不保证首次备选池全部有四领域资料。资料不足时应扩池或延期,而不是放松独立来源、年月精度、领域或封存标准。正式采集任务须先指定两位独立人工角色及存储访问方案;无人审资源就保持 `blocked`。 + +完成门(必须全部有证据): + +- [ ] 足量新人物通过 AA 原文、身份去重与历史曝光审计。 +- [ ] 每例事件数/年月精度/领域数达标,逐件独立来源核实,无拼造精度或强改领域。 +- [ ] 双人采集/复核职责真实独立,来源纠错保留前后关联。 +- [ ] 封存 manifest、持久分区、内容哈希和访问日志齐备;评分团队未读事件调参。 +- [ ] 打分实现、参数、半径/偏移、步长、并列规则和五项指标定义在揭晓前固定。 +- [ ] 首次全量运行与独立结果审阅完成,失败与限制不删;不自动打开确认门。 + +## 6. 遗留与对外边界 + +1. **真实用户申报偏差分布仍缺失**:需要知情授权、受控账号及独立出生记录的真实用户样本。用户采用候选、满意或认为“说得准”不能当真值;模拟名人偏移也不能替代该分布。后续另立隐私、抽样与标注任务,只发布聚合统计。 +2. **本轮未取得事件材料与人审资源**:本文件只有公开候选来源筛查。AA 原文归档、四领域/年月事件可得性、独立审阅与评分冻结都是未来采集任务的开工/交付条件,不在此冒写已完成。 +3. **来源网络限制**:原文直读返回校验页时,采用公开检索作备选资格线索;不绕过访问控制,不将线索当最终出生源审计。正式采集无法复核就阻塞该人物,不降级纳入。 +4. **数据集外推限制**:公开名人的资料密度、年龄、地区与事件报道方式不同于真实会话。即使 v5 通过,也只证明登记条件下的结果,不能推广到所有用户、更不能承诺唯一出生分钟。 +5. 本轮不再研究“加权重就能分开候选”或“簇合并导致整窗”这两条已被证伪的方向;见 [两轮结论页](rectification_minute_resolution_closure_2026_09_14.md) §6。始终先核口径,再看真值覆盖,不能以收窄区间换取挤出真值。 + +## Sources + +- 人物资格线索:§2 表内 26 个 Astro-Databank / Astro.com 原站链接(检索日期 2026-09-20;最终原文/人审复核尚未完成)。 +- 来源分级:[Astro-Databank Rodden Rating](https://www.astro.com/astro-databank/Help:RR)、[Data Source](https://www.astro.com/astro-databank/Help:DataSource)。 +- 仓库协议依据:[v4 构建协议](holdout_v4_build_2026_09_14.md)、[两轮研究结论](rectification_minute_resolution_closure_2026_09_14.md)、[BUG-098 / BUG-428](../BUG_HISTORY.md)。 diff --git a/docs/tasks/PROGRESS-rectification-validation-20260920.md b/docs/tasks/PROGRESS-rectification-validation-20260920.md new file mode 100644 index 00000000..4a8425d3 --- /dev/null +++ b/docs/tasks/PROGRESS-rectification-validation-20260920.md @@ -0,0 +1,73 @@ +# PROGRESS · 生时校正验证体系补缺(2026-09-20) + +## 基线与边界 + +- 执行基线:远端 `origin/staging = 5049e7847`,任务书基线 `fcad0637` 的后代。主检出保留原分支与未跟踪文件。 +- 工作树:`.worktrees/rectification-validation-20260920`;分支:`codex/rectification-validation-20260920`。 +- 一次性测试基线:`.worktrees/rectification-validation-baseline-20260920`(detached `5049e7847`)。 +- 仅离线评测、契约元数据与记录,不修改打分实现、生产行为、确认门,不提升 main、不改 workflow、不改 DNS。 +- 编号核对:开工 Bug 历史最大号 975;另一任务预留 976/977,本单使用 978–980。 + +## 决策与任务书偏离 + +完整核查 `BUG-098`、`BUG-318`、`BUG-427`、`BUG-428`。后两条明确规定已曝光案例不得重新算作独立盲测;任务书 T2 的“首次口径干净的官方盲测”没有在决策记录里授权推翻该红线。因此本轮只做**冻结当前实现的固定口径重跑**,不伪称新独立盲测,不提升官方盲测次数,不改发布六键和确认门。真正的独立验证仍需 T3 所定义的未来 v5;这一部分保持 blocked。 + +契约刷新需要相应更新旧测试中“当前实现必须等于历史 sidecar”的元数据断言;所有行为断言保留,原值/新值/原因在完成后逐项记录。 + +## 环境与基线实测 + +本机无项目 `.venv`;使用 `python`(Python 3.11.7,已有 pytest/swisseph)。`python3` launcher 退出 49。mcp / hypothesis 未安装;不顺带升级依赖。前端复用已安装 node_modules,工作树仅新增未跟踪依赖 junction,不纳入交付。 + +| 检查 | 基线结果 | 说明 | +| --- | --- | --- | +| `pre_work_check.py --remote-timeout 8 --command-timeout 45` | fail | 远端 verified;碎片扫描、适配器检查成功;focused tests 23 pass / 1 fail,镜像路径断言,已见于既有错误台账 | +| Python 定向 3 文件 | 21 pass / 1 fail | `test_minute_rectification_holdout_validator.py::test_frozen_implementation_hash_matches_manifest` 针对历史 v2 封存哈希失败,不改历史封存值 | +| 前端 `rectification-confirmation-gate.test.ts` | 7 pass / 0 fail | 后续按相同测试逐项比对 | +| Python quick | fail | 两侧均在 `interpretation_source_inventory_gate.py` 因缺 mcp 退出 1;46 条缺引用消息逐项相同 | +| 前端 tsc | 0 error | 基线与修改版均退出 0 | +| 前端 lint | 0 error / 120 warning | 基线与修改版相同,不顺带改 warning | +| 前端 npm test(默认并发) | 基线 3470 / 3387 pass / 83 fail | 修改版 3486 / 3409 pass / 77 fail;修改版失败标题为基线子集,0 新增失败;存在文件级环境/资源失败,另做串行补验 | +| 前端 build | fail | 两侧均为 Turbopack 拒绝指向工作树外的 node_modules junction;Static/gzip 环境缺口,未宣称通过 | + +基线/当前前端确认门定向均为 7/7,通过项逐条相同。默认并发全量的通过数量差异不是新增业务测试,只是部分文件级中断减少;不能宣称全部前端测试通过。两侧串行补验在数据库容器等待等长时阶段未产出最终汇总,主动停止,标记未完成而不是通过。完整失败标题对照已保存 `docs/testing/rectification-validation-20260920-results.json`(83 对 77,新增失败 0)。 + +额外 Python 安全定向:`test_rectification_confirmation_and.py`、`test_rectification_v5_vedastro_validation.py`、`test_repo_privacy_markers.py` 共 **85 passed**。补验清单见 `docs/testing/rectification-validation-20260920.md`。 + +## 独立验收与复核发现 + +- 字节级核对:12 个 `frozen_scoring.files` 全部与基线相同;确认门六个 runtime 键值及类型相同;生产 `decision_policy.py` / `sealed_holdout.py`、历史 `_candidate_moments()` 所在文件、`request_from_case()` 所在文件、主 API 与全部 `frontend/src` 均未改。 +- 任务书引用的 `frontend/src/app/page.tsx` 在执行基线已不存在(首页已迁移),因此用整个 `frontend/src` 字节对照证明无新增,不虚报旧路径行数。 +- 独立审查发现首版 T1 只在候选枚举保留跨日日期,矩阵下游的 transition proximity 仍使用统一日期,跨日候选得分有偏差。修复限定在离线组装层:按候选日期分组计算矩阵并回并;不改生产打分文件。首版统计不能作最终证据,须修复后重跑并补真实引擎回归。 +- 中途代理 API 余额错误导致执行和审查中断,用户要求继续后恢复;不把中断当任务完成。一次报告重定向被分类器按覆盖操作拒绝,经确认目标不存在后改为不覆盖的新建获准;另有子代理角色限制要求返回文档文本,由主会话汇总,不修改权限配置。 + +## 执行状态 + +- T1:修正版完整 900 组合已完成(20 例 × 15 偏移 × 3 半径,排除 0),JSON 与汇总表落盘。默认半径 ±15 在绝对偏移第 16 个整数分钟开始出窗;±30/60/120 对已测偏移几何包含率可恢复 100%,其中 ±120 仅几何推论、未评分。实际交付覆盖与唯一头名命中另列,不混称准确率。口径 v3、raman/mean、1 分钟、历史打分哈希 `b15d9ea15227cd58`;原生评分扩展哈希 `115c3fbcffdaff49`、研究适配哈希 `8d0c613dc0900adb`。 +- T2:当前实现冻结、20 例固定口径重跑、契约刷新与防复发测试完成;实现代理和独立审查均复算逐例/汇总一致。top-1/top-3(含并列)0.45/0.50、MAE 6.45 分钟,三项未达门槛;误确认率 0、稀疏证据拒绝率 1,确认覆盖 0。独立盲测资格仍 blocked,官方有效试次仍 0;不是新的发布成绩。哈希前缀 `b15d9ea15227cd58`、v3、raman/mean、±10、1 分钟。 +- T3:协议已完成。26 名公开 AA 来源候选,与 v3/v4 的 20 人交集为 0;未采事件、未完成人审、未建成 v5。来源直接访问有校验页限制,只算候选筛查,正式准入须复核 Source Notes。工时估算 86–172 人时。 +- T4:Bug 历史、BLOCKED、研究索引、任务索引、失败对照和补验清单已更新;2026-09-20 产品明确授权 push staging,本批进入交付。远端推送与部署分别核验,不把授权当完成。 + +## 最终验收结果 + +| 条目 | 结论 | 证据 | +| --- | --- | --- | +| T1 全量偏差评测、四项逐例输出、跨日修正、隐私字段 | 通过 | 900 组合、45 格,参数及评分/研究哈希齐全;汇总从逐例重算一致 | +| T2 当前实现冻结、重跑归属、契约新鲜度 | 通过 | 独立复算完整报告一致;新鲜度与拒绝漂移测试通过 | +| T2 新的独立官方盲测 | 未通过 / 资料条件缺口 | 已曝光样本不恢复独立性,官方试次保持 0;任务书措辞不覆盖 BUG-427/428 红线 | +| T2 指定相关 Python 总集 | 49 passed / 1 failed | 唯一失败是基线同现的历史 v2 哈希断言,错误哈希两侧逐字一致;未削弱断言 | +| 新增 T1 / T2 与 quick 桥接 | 28 passed | 含桥接重复收集的 14 项,不能算 28 项独立用例 | +| 确认门与隐私补充 | 通过 | Python 85 passed;前端确认门 7/7 与基线逐项相同;runtime 六键未变 | +| tsc / lint | 通过 | 两侧 0 error,lint 120 warning 相同 | +| quick / 全量前端 / 构建 | 环境及既有失败缺口 | 见上述基线对照;不可写全绿、Static 或 gzip 通过 | +| T3 v5 协议 | 通过(仅协议) | 26 人来源筛查、旧集交集 0、无出生分钟/坐标;v5 未建成,BUG-980 investigating | + +## 既有断言调整三栏 + +| 原值 | 新值 | 原因 | +| --- | --- | --- | +| Python 与前端 `current_tree_scorer.implementation_sha256` 绑定历史 post-audit sidecar | 绑定当前冻结重跑的 `frozen_record.implementation_sha256` | 元数据含义是当前树;继续绑旧报告强制过期。新增 freshness 同时验证当前源码,未削弱身份校验 | +| 官方 flag=false / trial_count=0、六 runtime 键、确认行为 | 原值不变 | 已曝光重跑不获得独立性;新增断言固定此边界 | + +## 交付说明 + +2026-09-20 产品明确要求「push 到 staging」。交付前 fetch 确认远端仍为 `5049e784`,本轮无上游冲突,BUG-978~980 无新增占用;本批实现与记录同次提交交付。推送后另以远端 SHA 核对结果为准;状态板“已合入”仅在远端核对后生效,不代表门禁或部署成功。本批含门禁路径,将触发自动质量门;本机完整 quick/构建缺口仍保留。没有动 main、workflow、DNS、生产资料、依赖和打分参数。 diff --git a/docs/tasks/README.md b/docs/tasks/README.md index 5b664bbf..27cfd7d5 100644 --- a/docs/tasks/README.md +++ b/docs/tasks/README.md @@ -290,7 +290,7 @@ | — (产品口头拍板,无任务书) | `PROGRESS-settings-ui-20260919.md` | **设置面板布局与资料入口整理**:基线 `4f4cd684`;四分区继续共用固定 `.settings-modal`,桌面导航 176px→200px 并加分隔,内容区增加内边距,表单 cap 440px→560px,导航移除误导性右箭头;账户头像 48px→56px;“添加其他人”移到分组标题操作区。已同步 `frontend/DESIGN.md` 与合同测试;tsc 0、lint 0 error、定向测试 21/21;build 被 Windows Skill runtime symlink 权限阻塞,浏览器走查待受控环境;BUG-970 保持 `investigating` | 待验收 | `codex/settings-ui-20260919` | | `TASK-owner-case-purge-20260919.md` | `PROGRESS-owner-case-purge-20260919.md` | **上游库主案例与本机路径残留清除(只含本仓)**:镜像同步带进库主本人案例(敏感案例标识与本机路径)并被 `origin/staging` 命中,涉及无引用的 `versions/` 三快照、前端 fixture、Python 测试、整机扫描台账、会话转录及上游 SKILL 快照。运行时无特判不用动。产品拍板整体删除不留匿名版;fixture 统一虚构常量;`import_yinduzhanxing.py` 加隐私排除项 + 新增仓库级隐私守卫测试;上游 SKILL 快照等库主清完再重导入(BLOCKED 记录)。上游仓的清理指令另见 `UPSTREAM-INSTRUCTION-owner-case-purge-20260919.md`(交给库主,不在本仓执行)。BUG-972/973 | **已验收通过(2026-09-20,`497798ac`);未合入 staging,等产品放行** | 两轮:首轮未通过(1 个隐私守卫冲突:新增路径规则与答案键守卫冲突,该守卫在快速门 glob 内,合入会让门禁红)→ 修复单 `TASK-owner-case-purge-fix2-20260919.md` → `497798ac` 通过。Claude 在 Linux 全依赖环境独立复验:快速门 Python 步 859 passed / 0 failed(上一轮就是这步红),Python 全量 63 红与基线逐条相同、0 新红,收集数与删除清单已记录;tsc 0 / lint 0 error、120 warning 同基线 / `npm test` 失败清单为基线子集(少 1 条工作流 YAML,非回归)/ `○ /` Static、首屏 gzip 与基线字节相同(前端自首轮提交起零改动)。上游指令文件已逐字节还原为基线原文;12 个已删测试名与计数已落进度记录;BLOCKED 两条已划掉。校正 Skill 哈希包、注册表、上游快照、`frontend/src`、主 API 全程 0 改动;库主本机用户名 0 命中。遗留 P3:进度记录和 BUG 状态文字待后续对账修正 | | `TASK-consult-smalltalk-fastpath-20260920.md` | — | **普通对话寒暄轮快速通道**:真机一句「你好」触发完整窗口排盘(活动面板「已完成 4 步」)+ `## 先回答你的问题` + 400 字判词 + 扣 1 点。三层叠加:`index.ts` 两处「every turn 必调排盘」(BUG-922)与 `contractReady()` 的 `requireTool` 把排盘变成硬合同;`product-voice.ts` OPENER SHAPE 标「三种模式共用」;唯一的 chit-chat 豁免句只在 `natalSpokenReportContract` 里、只拼进本命 Agent(BUG-977)。计费侧「写回复」与「扣点」绑在 `complete_consultation_response` 同一次调用,`cancel` 只退款不写消息,所以今天没有「不扣点但保留对话」的通道。**产品拍板 a:不扣点、不排盘、回一句白话**;**分流不得用正则/关键词/长度阈值**,改为进 Agent 之前一次极短的结构化模型调用(复用本轮已选模型),fail-open 一律落回完整路径;BUG-922/923 的三处合同一个字不改;新增 `complete_consultation_free` 迁移。BUG-976/977 | 待领取 | — | -| `TASK-rectification-validation-integrity-20260920.md` | — | **生时校正验证体系补缺(纯离线评测,不改打分不改产品)**:会议要求把「推断真实出生时间」与「用户认可的参考盘」分开证明。核对结论——**产品口径侧四条已落地**(`accepted`≠`confirmed` 两条写入路径、确认门 fail-closed 且 `holdout` 为 `not_ready` 使 `confirmation_allowed` 不可能为真、采用不写 `reported_birth_time`、无任何把采用率当准确率的指标;运行时也无按生日走捷径的分支);**缺口全在评测本身**。三条:① 封存契约 `rectification_sealed_holdout.v1.json` 的三个打分哈希互不相同(封存 `f41c298d` / 契约记录 `99730c84` / 基线实测 `b15d9ea1`),`official_eval_trial_count: 0`——当前实现**从未产出过一次有效官方盲测**,唯一跑过那次已被资料审计作废(top-1 `0.15`,发布门要 `0.60`),可见的 `0.45` 自带「不得当发布指标」标记(BUG-978);② 全部离线评测的候选窗**以真值为圆心**(`_candidate_moments()`、`request_from_case()` 的 `true_time`),生产以申报时间为圆心(`ENGINE_SEARCH_RADIUS_MINUTES = 15`)——「真值掉出窗外」这一失败模式从不可见(BUG-979);③ v3 封存但每例仅 3 事件、v4 有 7+ 事件却已被看过并用于调参,**无口径干净又贴近真实会话的封存集**;六题回放的 `0.80/0.55/0.35` 是「真值方向最优答」的上帝视角上界,±30/±60 仍低于发布门(BUG-980)。T1 申报偏差敏感性 sweep、T2 有效重跑 + 契约对齐 + **防复发新测试**、T3 只出 v5 采集协议、T4 记录。**硬红线:不得改 12 个打分文件、不得用封存集调参、不得把 `status` 改 `ready`。** 家庭信息(父母职业/兄弟姐妹)在拿到基线数字前不开工——现有七领域全是带日期事件,静态属性没有输入口。前置:owner-case-purge 三提交仍未合入 staging。BUG-978~980 | 待领取 | — | +| `TASK-rectification-validation-integrity-20260920.md` | `PROGRESS-rectification-validation-20260920.md` | **生时校正验证体系补缺(纯离线评测,不改打分不改产品)**:会议要求把「推断真实出生时间」与「用户认可的参考盘」分开证明。核对结论——**产品口径侧四条已落地**(`accepted`≠`confirmed` 两条写入路径、确认门 fail-closed 且 `holdout` 为 `not_ready` 使 `confirmation_allowed` 不可能为真、采用不写 `reported_birth_time`、无任何把采用率当准确率的指标;运行时也无按生日走捷径的分支);**缺口全在评测本身**。三条:① 封存契约 `rectification_sealed_holdout.v1.json` 的三个打分哈希互不相同(封存 `f41c298d` / 契约记录 `99730c84` / 基线实测 `b15d9ea1`),`official_eval_trial_count: 0`——当前实现**从未产出过一次有效官方盲测**,唯一跑过那次已被资料审计作废(top-1 `0.15`,发布门要 `0.60`),可见的 `0.45` 自带「不得当发布指标」标记(BUG-978);② 全部离线评测的候选窗**以真值为圆心**(`_candidate_moments()`、`request_from_case()` 的 `true_time`),生产以申报时间为圆心(`ENGINE_SEARCH_RADIUS_MINUTES = 15`)——「真值掉出窗外」这一失败模式从不可见(BUG-979);③ v3 封存但每例仅 3 事件、v4 有 7+ 事件却已被看过并用于调参,**无口径干净又贴近真实会话的封存集**;六题回放的 `0.80/0.55/0.35` 是「真值方向最优答」的上帝视角上界,±30/±60 仍低于发布门(BUG-980)。T1 申报偏差敏感性 sweep、T2 有效重跑 + 契约对齐 + **防复发新测试**、T3 只出 v5 采集协议、T4 记录。**硬红线:不得改 12 个打分文件、不得用封存集调参、不得把 `status` 改 `ready`。** 家庭信息(父母职业/兄弟姐妹)在拿到基线数字前不开工——现有七领域全是带日期事件,静态属性没有输入口。前置:owner-case-purge 三提交仍未合入 staging。BUG-978~980 | 已合入(本批远端 SHA 核对后生效);独立盲测仍 blocked,门禁/部署待核验 | `codex/rectification-validation-20260920`;900 组合、当前冻结重跑、v5 协议完成;新增/桥接 28 pass,相关总集49 pass/1基线失败,确认门不变;2026-09-20 产品授权 push staging,同批交付,详见进度 | ## 命名与归档 diff --git a/docs/testing/rectification-validation-20260920-results.json b/docs/testing/rectification-validation-20260920-results.json new file mode 100644 index 00000000..4ea80668 --- /dev/null +++ b/docs/testing/rectification-validation-20260920-results.json @@ -0,0 +1,193 @@ +{ + "baseline_commit": "5049e7847", + "scope": "frontend_default_parallel_npm_test", + "counts": { + "baseline": { + "tests": 3470, + "pass": 3387, + "fail": 83, + "skipped": 0 + }, + "current": { + "tests": 3486, + "pass": 3409, + "fail": 77, + "skipped": 0 + } + }, + "failure_titles": { + "baseline": [ + "Better Auth supports shared user OTP/password sessions for admins", + "Owner recovery grants only one currently loginable synced identity admin", + "PR-4 candidate decisions use server UUIDs, receipt-derived gates and separate acceptance/confirmation", + "a multi-domain plan carries every executed route's checklist once", + "a route the skill declares no checklist for is reported, not filled in with another route's", + "admin code functions reject immutable codes, revoked redemption, and roll back on audit failure", + "admin customer reset clears only rebuildable application state", + "append_consultation_question ignores thinking fields and enforces the physical JSON cap", + "billing order adjustments and redemption reasons are atomic and audited", + "billing, subscriptions, usage authorization, RBAC, and model publication remain transactional", + "block_scan RPCs are service_role-only and advance a declared period", + "candidate scores stay out of the specified client ownership boundary", + "changed staging workflows are syntactically valid YAML", + "checked-in registry verifies hashed product packages and leaves consult on the live skill", + "creates every absent backup path component privately despite a permissive caller umask", + "database drops secret refs, generates immutable codes, and invalidates evidence after key rotation", + "database env validator accepts a private valid file without printing values", + "database env validator accepts punctuated literal required secrets", + "database env validator enforces an explicit staging owner uid without printing values", + "database env validator rejects symlinks and unsafe modes", + "document v2 and durable job migrations are exact db/supabase mirrors", + "document v2 migration preserves v1 reads and adds explicit product depth", + "durable job migration atomically enqueues reports and exposes lease RPCs only to service_role", + "durable job migration defines the complete state, lease, retry, and identity contract", + "every color token used as a Tailwind utility is exposed through @theme", + "find enumeration failures preserve existing backups and do not report completion", + "first immutable deployment rolls back to validated local image IDs", + "further reading offers the references the skill names, and only ones that exist", + "general agent runtime has no Jyotish skill package and no personal chart tool", + "health uses the skill's health-timing-strict checklist", + "immutable Skill migration enforces RPC-only identity, legacy adoption, and cascade-safe receipts", + "ingest P0: education kinds, batch confirm, opening focus reuse, precision lock", + "is-docs-only-range.sh decides from local history and refuses non-ancestor ranges", + "job migration mirror is exact and preserves personal_reports as the document projection", + "live consult skill reads a hand-updated tree without a registry hash", + "live staging sync preserves env, state, incoming files, and encrypted backups", + "live staging sync repairs nested deploy-tree drift without preserving foreign ownership", + "local PostgreSQL applies the reviewed business schema and serves authenticated business calls", + "longform appendices are owner-read, service-written, and never change report status", + "no plan can spend the answer's context on method", + "operation-level admin email reauthentication is removed from routes and UI", + "path traversal and symlink escape fail closed", + "personal Agent exposes the Jyotish Skill and named server tool", + "personal report job migration is atomic, lease-bound, recoverable, and owner read-only", + "personal report sections enforce owner-read RLS and service-owned durable transitions", + "personal_reports.card_summary is nullable, owner-read, service-written, and length-capped", + "production env validators accept only self-hosted production selectors and role URLs", + "public code never imports the internal dynamic choice contract", + "read_report_candidate_range is service_role-only, returns only the window, and stays empty without rows", + "rectification agent maps setup failures without calling the rest of the handler", + "redeem security: case-sensitive hashing, rate limiting, idempotency and order ownership", + "refuses full disks and removes a failed-pipeline partial file", + "rejects a direct canonical sticky shared backup directory before chmod", + "rejects destructive backup directory aliases and symlink components before mutation", + "rejects unsafe writable backup parents before creating the target", + "same-second backups publish once without overwriting the completed archive", + "saving a published product forks a draft and delete retires or removes it", + "self-hosted identity migration creates Better Auth tables with least privilege", + "service and restricted admin database identities stay separated", + "staging backups are encrypted, atomic, private, and retain the newest three", + "staging env validator rejects selector drift, duplicates, and unsafe permissions", + "staging model provider env preparation removes legacy settings and keeps one stable key", + "symbolic links are rejected even when their target stays inside the project root", + "tests\\\\birth-time-guide-agent.test.ts", + "tests\\\\consultation-workflow-request.test.ts", + "tests\\\\rectification-confirmation-gate.test.ts", + "tests\\\\rectification-settled-render-split.test.ts", + "tests\\\\rectification-skipped-health-deadend-20260909.test.ts", + "tests\\\\rectification-split-holdout.test.ts", + "tests\\\\rectification-spoken-prompt.test.ts", + "tests\\\\rectification-v9-agent.test.ts", + "tests\\\\skill-binding.test.ts", + "the delivered method is quoted from the live skill tree", + "the route's own strict checklist reaches the answer, not just the package listing", + "the widest legal plan still fits the budget", + "v9 agent api migration applies, seeds the runtime flag and guards consent", + "v9 enforces profile gating, ownership and terminal read-only", + "v9 evidence lifecycle: quote grounding, idempotency, confirm and revision lineage", + "v9 ignores a historical active minute and allows reselection within the live result", + "v9 legacy backfill maps statuses, keeps one resumable per user and is idempotent", + "v9 migration applies on a fresh database and re-applies idempotently", + "v9 open is atomic, idempotent and allows separate homepage cases", + "wealth uses finance-timing-strict as the live checklist, with the wealth alias in the heading" + ], + "current": [ + "Better Auth supports shared user OTP/password sessions for admins", + "Owner recovery grants only one currently loginable synced identity admin", + "PR-4 candidate decisions use server UUIDs, receipt-derived gates and separate acceptance/confirmation", + "a multi-domain plan carries every executed route's checklist once", + "a route the skill declares no checklist for is reported, not filled in with another route's", + "admin code functions reject immutable codes, revoked redemption, and roll back on audit failure", + "admin customer reset clears only rebuildable application state", + "append_consultation_question ignores thinking fields and enforces the physical JSON cap", + "billing order adjustments and redemption reasons are atomic and audited", + "billing, subscriptions, usage authorization, RBAC, and model publication remain transactional", + "block_scan RPCs are service_role-only and advance a declared period", + "candidate scores stay out of the specified client ownership boundary", + "changed staging workflows are syntactically valid YAML", + "checked-in registry verifies hashed product packages and leaves consult on the live skill", + "creates every absent backup path component privately despite a permissive caller umask", + "database env validator accepts a private valid file without printing values", + "database env validator accepts punctuated literal required secrets", + "database env validator enforces an explicit staging owner uid without printing values", + "database env validator rejects symlinks and unsafe modes", + "document v2 and durable job migrations are exact db/supabase mirrors", + "document v2 migration preserves v1 reads and adds explicit product depth", + "durable job migration atomically enqueues reports and exposes lease RPCs only to service_role", + "durable job migration defines the complete state, lease, retry, and identity contract", + "every color token used as a Tailwind utility is exposed through @theme", + "find enumeration failures preserve existing backups and do not report completion", + "first immutable deployment rolls back to validated local image IDs", + "further reading offers the references the skill names, and only ones that exist", + "general agent runtime has no Jyotish skill package and no personal chart tool", + "health uses the skill's health-timing-strict checklist", + "immutable Skill migration enforces RPC-only identity, legacy adoption, and cascade-safe receipts", + "ingest P0: education kinds, batch confirm, opening focus reuse, precision lock", + "is-docs-only-range.sh decides from local history and refuses non-ancestor ranges", + "job migration mirror is exact and preserves personal_reports as the document projection", + "live consult skill reads a hand-updated tree without a registry hash", + "live staging sync preserves env, state, incoming files, and encrypted backups", + "live staging sync repairs nested deploy-tree drift without preserving foreign ownership", + "local PostgreSQL applies the reviewed business schema and serves authenticated business calls", + "longform appendices are owner-read, service-written, and never change report status", + "no plan can spend the answer's context on method", + "operation-level admin email reauthentication is removed from routes and UI", + "path traversal and symlink escape fail closed", + "personal Agent exposes the Jyotish Skill and named server tool", + "personal report job migration is atomic, lease-bound, recoverable, and owner read-only", + "personal report sections enforce owner-read RLS and service-owned durable transitions", + "personal_reports.card_summary is nullable, owner-read, service-written, and length-capped", + "production env validators accept only self-hosted production selectors and role URLs", + "public code never imports the internal dynamic choice contract", + "read_report_candidate_range is service_role-only, returns only the window, and stays empty without rows", + "rectification agent maps setup failures without calling the rest of the handler", + "redeem security: case-sensitive hashing, rate limiting, idempotency and order ownership", + "refuses full disks and removes a failed-pipeline partial file", + "rejects a direct canonical sticky shared backup directory before chmod", + "rejects destructive backup directory aliases and symlink components before mutation", + "rejects unsafe writable backup parents before creating the target", + "same-second backups publish once without overwriting the completed archive", + "saving a published product forks a draft and delete retires or removes it", + "self-hosted identity migration creates Better Auth tables with least privilege", + "service and restricted admin database identities stay separated", + "staging backups are encrypted, atomic, private, and retain the newest three", + "staging env validator rejects selector drift, duplicates, and unsafe permissions", + "staging model provider env preparation removes legacy settings and keeps one stable key", + "symbolic links are rejected even when their target stays inside the project root", + "tests\\\\birth-time-guide-agent.test.ts", + "tests\\\\consultation-workflow-request.test.ts", + "tests\\\\rectification-v9-agent.test.ts", + "tests\\\\skill-binding.test.ts", + "the delivered method is quoted from the live skill tree", + "the route's own strict checklist reaches the answer, not just the package listing", + "the widest legal plan still fits the budget", + "v9 agent api migration applies, seeds the runtime flag and guards consent", + "v9 enforces profile gating, ownership and terminal read-only", + "v9 evidence lifecycle: quote grounding, idempotency, confirm and revision lineage", + "v9 ignores a historical active minute and allows reselection within the live result", + "v9 legacy backfill maps statuses, keeps one resumable per user and is idempotent", + "v9 migration applies on a fresh database and re-applies idempotently", + "v9 open is atomic, idempotent and allows separate homepage cases", + "wealth uses finance-timing-strict as the live checklist, with the wealth alias in the heading" + ] + }, + "new_failures": [], + "not_claiming_full_pass": true, + "serial_followup": "stopped_without_final_summary_while_database_fixture_waited", + "build_both": "Turbopack rejects external node_modules junction", + "tsc_both_exit_code": 0, + "lint_both": { + "errors": 0, + "warnings": 120 + } +} diff --git a/docs/testing/rectification-validation-20260920.md b/docs/testing/rectification-validation-20260920.md new file mode 100644 index 00000000..847d01d1 --- /dev/null +++ b/docs/testing/rectification-validation-20260920.md @@ -0,0 +1,24 @@ +# 生时校正验证补缺 · 补验清单(2026-09-20) + +本轮无产品/界面/数据库行为改动。不要让用户提供私人出生资料以补本轮验收;偏移评测使用已登记公开数据。 + +## 在完整依赖的 Linux 环境补验 + +1. 用本轮提交创建干净工作树,安装仓库锁定依赖,不借用其他账号/凭据。 +2. 运行 `python scripts/pre_work_check.py --remote-timeout 8 --command-timeout 45`。本机已知基线碎片镜像断言失败,不能以远端 verified 代替预检通过。 +3. 运行 `python -m pytest tests/test_reported_offset_research.py tests/test_sealed_holdout_contract_freshness.py tests/test_rectification_validation_integrity_gate.py tests/test_rectification_confirmation_and.py tests/test_minute_rectification_holdout_validator.py tests/test_minute_rectification_fact_blind_eval_v4.py -q`。旧 v2 封存哈希测试在基线已失败,不改历史值来消除失败。 +4. 运行 `python scripts/run_quality_gate.py --profile quick`;Windows 本机在 source inventory 检查因缺 mcp 阻塞。完整输出与同基线逐项比对,不把“新增回归通过”写成“整个快速门通过”。 +5. 前端运行 `tsc --noEmit`、`npm run lint`、`npm test`、`npm run build`;单独确认 `rectification-confirmation-gate.test.ts` 全绿。构建必须得到完整 `/` Static 与首屏 gzip 产物再比较,不能以未修改业务代码替代。 +6. 核对冻结的 12 个打分文件与执行基线字节相同,`status=not_ready`、`confirmation_coverage_rate=0`、官方独立盲测次数仍为零。 + +## 交付后才执行 + +- 获准推 staging 后,检查 Gitea 的对应 SHA 门禁和部署;不参考只读 GitHub 镜像。 +- 核对 `/api/health` 的 `deployment.gitCommit` 为最近含门禁路径改动的 staging SHA;若之后仅文档提交,另跑 docs-only-range 验证。 +- 未部署不得写“线上验证通过”。本轮不提升 main。 + +## 研究遗留(不是软件环境补齐就能解决) + +- 新独立盲测必须等待未曝光 v5 的独立标注、人审、封存与预注册;v3 重跑不能替代。 +- 真实申报偏差分布需要受控用户样本、独立出生记录与合规授权。本轮敏感性表不代表真实用户总体表现。 +- v5 备选人物只是候选筛选,不是资料已经采集完毕或人审已通过。 diff --git a/frontend/tests/rectification-confirmation-gate.test.ts b/frontend/tests/rectification-confirmation-gate.test.ts index d59fcf2b..b662504b 100644 --- a/frontend/tests/rectification-confirmation-gate.test.ts +++ b/frontend/tests/rectification-confirmation-gate.test.ts @@ -81,6 +81,13 @@ const diagnosticReport = JSON.parse(readFileSync( verified_minute_claim_allowed: boolean; }; }; +const currentTreeRerun = JSON.parse(readFileSync( + new URL("../../docs/research/sealed_holdout_rerun_2026_09_20.json", import.meta.url), + "utf8", +)) as { + frozen_record: { implementation_sha256: string }; + official_valid_independent_blind: boolean; +}; type ProductHoldoutContract = typeof SEALED_MINUTE_HOLDOUT & { metrics_produced_by: { implementation_sha256: string; @@ -190,10 +197,16 @@ test("sealed holdout aggregates match the v3 report and still block confirmation productHoldout.metrics_produced_by.source_report, "references/real_case_calibration/minute_rectification_holdout_v3_report.json", ); + // 原值: 历史 sidecar 哈希 / 新值: 当前冻结重跑哈希 / 原因: BUG-978 身份新鲜度,不改变确认门。 assert.equal( productHoldout.current_tree_scorer.implementation_sha256, - diagnosticReport.frozen_scoring.actual_sha256, + currentTreeRerun.frozen_record.implementation_sha256, ); + assert.equal( + productHoldout.current_tree_scorer.source_report, + "docs/research/sealed_holdout_rerun_2026_09_20.json", + ); + assert.equal(currentTreeRerun.official_valid_independent_blind, false); assert.equal(productHoldout.current_tree_scorer.matches_metrics_scorer, false); assert.equal(productHoldout.current_tree_scorer.official_eval_implementation_hash_matches, false); assert.equal(productHoldout.current_tree_scorer.official_eval_trial_count, diagnosticReport.trial_count); diff --git a/references/rectification_sealed_holdout.v1.json b/references/rectification_sealed_holdout.v1.json index aecebb18..ab2ff8e3 100644 --- a/references/rectification_sealed_holdout.v1.json +++ b/references/rectification_sealed_holdout.v1.json @@ -8,8 +8,10 @@ "previous_pilot_id": "minute_rectification_holdout_v2", "dataset_path": "references/real_case_calibration/minute_rectification_holdout_v3.json", "report_path": "references/real_case_calibration/minute_rectification_holdout_v3_report.json", - "source_audit_status": "invalidated_after_replay", - "evaluated_on": "2026-07-21", + "source_audit_status": "corrected_known_date_errors", + "evaluated_on": "2026-09-20", + "evaluated_on_scope": "latest_fixed_protocol_rerun_not_historical_runtime_metrics", + "historical_source_audit_status": "invalidated_after_replay", "report_status": "invalidated_after_source_audit", "trial_count": 20, "top_3_rate": 0.25, @@ -22,11 +24,40 @@ "evaluated_on": "2026-07-21" }, "current_tree_scorer": { - "implementation_sha256": "99730c84c6434f52669a05e4d9a4a87df0a218e3237b5436b933252f2028384e", + "implementation_sha256": "b15d9ea15227cd58b3097555537fa7c37fb5a993905ada4f5aa04f52de1f8f18", "matches_metrics_scorer": false, "official_eval_implementation_hash_matches": false, "official_eval_trial_count": 0, - "source_report": "references/real_case_calibration/minute_rectification_holdout_v3_post_audit_diagnostic_report.json" + "source_report": "docs/research/sealed_holdout_rerun_2026_09_20.json", + "fixed_protocol_rerun_trial_count": 20, + "fixed_protocol_rerun_hash_matches": true, + "metrics": { + "top_1_rate": 0.45, + "top_3_rate": 0.5, + "mean_absolute_minute_error": 6.45, + "false_confirmation_rate": 0.0, + "correct_insufficient_evidence_rejection_rate": 1.0, + "confirmation_coverage_rate": 0.0 + } + }, + "current_tree_fixed_protocol_rerun": { + "evaluated_on": "2026-09-20", + "report_path": "docs/research/sealed_holdout_rerun_2026_09_20.json", + "freeze_record_path": "docs/research/sealed_holdout_rerun_2026_09_20.freeze.json", + "implementation_sha256": "b15d9ea15227cd58b3097555537fa7c37fb5a993905ada4f5aa04f52de1f8f18", + "scorer_frozen_before_rerun": true, + "source_audit_status": "corrected_known_date_errors", + "trial_count": 20, + "official_valid_independent_blind": false, + "is_blind_evaluation": false, + "truth_hidden_from_ranker": true, + "results_previously_seen": true, + "must_not_claim_as_release_metrics": true, + "must_not_use_for_tuning": true, + "verified_minute_claim_allowed": false, + "metric_gates_passed": false, + "events_per_case": 3, + "boundary": "First current-hash fixed-protocol rerun in this task, not a first independent official blind evaluation. Three-event low-information protocol is not representative of real sessions and is not a mathematical accuracy lower bound. Historical v3/v4 score exposure requires a fresh sealed set." }, "current_tree_unfrozen_diagnostic": { "is_blind_evaluation": false, diff --git a/scripts/research/reported_offset_sweep.py b/scripts/research/reported_offset_sweep.py new file mode 100644 index 00000000..4f1a2429 --- /dev/null +++ b/scripts/research/reported_offset_sweep.py @@ -0,0 +1,211 @@ +#!/usr/bin/env python3 +"""Reported-centre sensitivity on exposed public cases, without oracle answers.""" +from __future__ import annotations + +import argparse +import json +import sys +from datetime import datetime, timedelta +from pathlib import Path +from typing import Any + +ROOT = Path(__file__).resolve().parents[2] +if str(ROOT) not in sys.path: + sys.path.insert(0, str(ROOT)) + +from scripts.active_rectification_event_engine import ( + AYANAMSA, NODE_MODE, compute_candidate_static_contexts, +) +from scripts.minute_rectification_blind_eval import _opaque_winner, implementation_sha256 +from scripts.minute_rectification_holdout_validator import validate +from scripts.rectification.candidate_contrast import select_signature_representatives +from scripts.rectification.scoring_service import build_event_contribution_matrix, score_from_matrix +from scripts.research.cluster_width_lib import SEPARATION_LEAD, still_valid_public +from scripts.research.probe_supply_after_six import request_from_case +from scripts.research.sealed_holdout_rerun import DATASET, file_sha256, opaque_order + +OFFSETS = (-30, -20, -15, -10, -8, -5, -3, 0, 3, 5, 8, 10, 15, 20, 30) +RADII = (15, 30, 60) +MINUTE_STEP = 1 +PRODUCTION_FILES = [ + "scripts/rectification/scoring_service.py", + "scripts/rectification/dasha_transition_proximity.py", + "scripts/rectification/candidate_contrast.py", + "scripts/rectification/case_holdout.py", + "scripts/rectification/contracts.py", + "scripts/rectification/event_probes.py", +] +RESEARCH_FILES = [ + "scripts/research/reported_offset_sweep.py", + "scripts/research/cluster_width_lib.py", + "scripts/research/probe_supply_after_six.py", + "scripts/research/sealed_holdout_rerun.py", + "scripts/minute_rectification_blind_eval.py", +] + + +def shifted_window(case: dict[str, Any], offset: int, radius: int) -> tuple[dict[str, Any], list[datetime]]: + if radius not in (*RADII, 120): + raise ValueError("unsupported_radius") + true_at = datetime.fromisoformat(f"{case['birth']['date']}T{case['birth']['time']}:00") + reported_at = true_at + timedelta(minutes=offset) + start = reported_at - timedelta(minutes=radius) + end = reported_at + timedelta(minutes=radius) + candidates = [start + timedelta(minutes=i) for i in range(0, radius * 2 + 1, MINUTE_STEP)] + # Reuse only the established event normalization, then replace the window. + # The ranker sees neither the truth label nor the simulated offset. + request = request_from_case(case) + request.update({ + "birth_date": start.date().isoformat(), + "start_time": start.strftime("%H:%M"), + "end_time": end.strftime("%H:%M"), + "minute_step": MINUTE_STEP, + "ayanamsa": AYANAMSA, + "node_mode": NODE_MODE, + }) + return request, candidates + + +def score_window(request: dict[str, Any], contexts: list[dict[str, Any]]) -> list[dict[str, Any]]: + """Keep auxiliary transition anchors on each candidate's actual date. + + The production matrix's transition-proximity helper accepts one birth date + per call, unlike the static chart layer which reads candidate_at. Grouping + is only an offline adapter; it does not change the production scorer. + """ + by_date: dict[str, list[dict[str, Any]]] = {} + for context in contexts: + by_date.setdefault(context["candidate_at"].date().isoformat(), []).append(context) + by_time = {} + for candidate_date, group in by_date.items(): + dated_request = {**request, "birth_date": candidate_date} + built = build_event_contribution_matrix(dated_request, static_contexts=group) + by_time.update({row["time"]: row for row in score_from_matrix(dated_request, built)}) + return [by_time[context["candidate_at"].strftime("%H:%M")] for context in contexts] + + +def delivery_moments(public: list[dict[str, Any]], candidates: list[datetime]) -> list[datetime]: + """Initial delivery envelope in date-aware order; no truth-based narrowing.""" + valid = still_valid_public(public, {}, lead=SEPARATION_LEAD) + clocks = {str(value)[:5] for row in valid for value in (row.get("cluster_times") or [row["time"]])} + return [candidate for candidate in candidates if candidate.strftime("%H:%M") in clocks] + + +def reveal_metrics( + rows: list[dict[str, Any]], candidates: list[datetime], delivery: list[datetime], + truth: datetime, benchmark_id: str, case_id: str, +) -> dict[str, Any]: + ordered = opaque_order(benchmark_id, case_id, rows) + predicted_clock = _opaque_winner(benchmark_id, case_id, rows) + by_clock = {candidate.strftime("%H:%M"): candidate for candidate in candidates} + if len(by_clock) != len(candidates): + raise ValueError("ambiguous_candidate_clock") + predicted = by_clock[predicted_clock] + within = min(candidates) <= truth <= max(candidates) + in_candidates = truth in candidates + rank = next((i for i, row in enumerate(ordered, 1) if by_clock[row["time"]] == truth), None) + covered = bool(delivery) and min(delivery) <= truth <= max(delivery) + return { + "truth_in_window": within, + "truth_in_candidates": in_candidates, + "true_rank": rank, + "top_1_hit": predicted == truth, + "top_1_minute_error": abs((predicted - truth).total_seconds()) / 60, + "delivery_covers_truth": covered, + "candidate_count": len(candidates), + "delivery_width_minutes": int((max(delivery) - min(delivery)).total_seconds() / 60) + 1 if delivery else 0, + } + + +def summarize(trials: list[dict[str, Any]], radii: tuple[int, ...], offsets: tuple[int, ...]) -> list[dict[str, Any]]: + result = [] + for radius in radii: + for offset in offsets: + group = [row for row in trials if row["radius_minutes"] == radius and row["offset_minutes"] == offset] + count = len(group) + result.append({ + "radius_minutes": radius, "offset_minutes": offset, "trial_count": count, + **{key: round(sum(row[source] for row in group) / count, 4) if count else None for key, source in ( + ("truth_in_window_rate", "truth_in_window"), + ("top_1_rate", "top_1_hit"), + ("delivery_coverage_rate", "delivery_covers_truth"), + ("mean_absolute_minute_error", "top_1_minute_error"), + )}, + }) + return result + + +def run(dataset: Path = DATASET, radii: tuple[int, ...] = RADII, offsets: tuple[int, ...] = OFFSETS) -> dict[str, Any]: + manifest = json.loads(dataset.read_text(encoding="utf-8")) + frozen_files = manifest["frozen_scoring"]["files"] + scoring_files = sorted(set(frozen_files + PRODUCTION_FILES)) + starting_hash = implementation_sha256(scoring_files) + validation = validate(dataset) + invalid = validation["invalid_cases"] + trials = [] + for index, case in enumerate(manifest["cases"], 1): + if case["case_id"] in invalid: + continue + # Reuse static chart calculations, not window-dependent scores or ranks. + contexts_by_moment: dict[datetime, dict[str, Any]] = {} + for radius in radii: + for offset in offsets: + request, candidates = shifted_window(case, offset, radius) + missing = [moment for moment in candidates if moment not in contexts_by_moment] + if missing: + contexts_by_moment.update(zip(missing, compute_candidate_static_contexts(request, candidates=missing), strict=True)) + contexts = [contexts_by_moment[moment] for moment in candidates] + rows = score_window(request, contexts) + public = select_signature_representatives(rows, contexts) + delivery = delivery_moments(public, candidates) + # All scores and the delivery envelope are locked before reveal. + truth = datetime.fromisoformat(f"{case['birth']['date']}T{case['birth']['time']}:00") + trials.append({ + "case_ordinal": index, "offset_minutes": offset, "radius_minutes": radius, + **reveal_metrics(rows, candidates, delivery, truth, manifest["benchmark_id"], case["case_id"]), + }) + if implementation_sha256(scoring_files) != starting_hash: + raise ValueError("scorer_changed_during_sweep") + return { + "scope": "reported_offset_sensitivity_not_product_accuracy", + "specification": { + "ayanamsa": AYANAMSA, "node_mode": NODE_MODE, + "radii_minutes": list(radii), "offsets_minutes": list(offsets), "minute_step": MINUTE_STEP, + "dataset": dataset.relative_to(ROOT).as_posix(), "dataset_sha256": file_sha256(dataset), + "dataset_benchmark_id": manifest["benchmark_id"], + "implementation_sha256": implementation_sha256(frozen_files), + "implementation_sha256_prefix": implementation_sha256(frozen_files)[:16], + "production_scoring_files": scoring_files, "production_scoring_sha256": starting_hash, + "research_files": RESEARCH_FILES, + "research_implementation_sha256": implementation_sha256(RESEARCH_FILES), + "hash_scope": "explicit_identity_file_sets_not_a_transitive_dependency_lock", + "scorer": "native_event_contribution_matrix_not_shadow_fact_ranker", + "delivery": "initial_signature_clusters_peak_gap_lt_8_envelope_no_answers_no_elimination", + "rank": "score_desc_then_sha256(benchmark_id:case_id:candidate_time)", + "grid": "every_minute_inclusive_not_production_two_minute_sampling", + "cross_midnight": "date_aware_candidates_distance_and_matrix_grouped_by_candidate_date", + "evaluator_sha256": file_sha256(Path(__file__)), + "replay_revision": "candidate_date_grouped_v2", + "supersedes": "initial_sweep_invalidated_cross_midnight_transition_anchor", + "is_blind_evaluation": False, "truth_hidden_from_ranker": True, + "results_previously_seen": True, "must_not_use_for_tuning": True, + }, + "excluded_cases": invalid, "case_count": validation["valid_public_aa_cases"], + "trial_count": len(trials), "trials": trials, + "summary": summarize(trials, radii, offsets), + "widening_geometry": { + "scored_radii": list(radii), + "radius_120_scored": 120 in radii, + "truth_in_window_condition": "abs(reported_offset_minutes) <= radius_minutes", + "radius_15_first_integer_minute_outside": 16, + "all_tested_offsets_within_radius_30_60_120": max(map(abs, offsets)) <= 30, + "boundary": "Geometry rescues candidate inclusion only, not ranking or delivery coverage; no population frequency without a reported-offset distribution.", + }, + } + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--json", action="store_true") + args = parser.parse_args() + print(json.dumps(run(), ensure_ascii=False, indent=2)) diff --git a/scripts/research/sealed_holdout_rerun.py b/scripts/research/sealed_holdout_rerun.py new file mode 100644 index 00000000..071655d2 --- /dev/null +++ b/scripts/research/sealed_holdout_rerun.py @@ -0,0 +1,153 @@ +#!/usr/bin/env python3 +"""Freeze and replay the exposed v3 corpus; never claim a fresh blind holdout.""" +from __future__ import annotations + +import argparse +import hashlib +import json +import sys +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +ROOT = Path(__file__).resolve().parents[2] +if str(ROOT) not in sys.path: + sys.path.insert(0, str(ROOT)) + +from scripts.active_rectification_event_engine import AYANAMSA, NODE_MODE +from scripts.minute_rectification_blind_eval import ( + _candidate_moments, _clock_distance, _opaque_winner, _request, + implementation_sha256, summarize_trials, +) +from scripts.minute_rectification_fact_blind_eval_v4 import _would_confirm +from scripts.minute_rectification_fact_ranker_v4 import ( + ALGORITHM_VERSION, rank_fact_rows, score_fact_ranker_v4, +) +from scripts.minute_rectification_feature_facts_v4 import build_feature_fact_rows +from scripts.minute_rectification_holdout_validator import validate + +DATASET = ROOT / "references/real_case_calibration/minute_rectification_holdout_v3.json" +FREEZE = ROOT / "docs/research/sealed_holdout_rerun_2026_09_20.freeze.json" +REPORT = ROOT / "docs/research/sealed_holdout_rerun_2026_09_20.json" + + +def file_sha256(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def opaque_order(benchmark_id: str, case_id: str, rows: list[dict[str, Any]]) -> list[dict[str, Any]]: + """Extend the existing opaque winner to a total, truth-independent ranking.""" + return sorted(rows, key=lambda row: ( + -row["score"], + hashlib.sha256(f"{benchmark_id}:{case_id}:{row['time']}".encode()).hexdigest(), + )) + + +def freeze_record(dataset: Path = DATASET) -> dict[str, Any]: + manifest = json.loads(dataset.read_text(encoding="utf-8")) + files = manifest["frozen_scoring"]["files"] + return { + "record_version": "exposed-v3-fixed-protocol-rerun-v1", + "frozen_at_utc": datetime.now(timezone.utc).isoformat(), + "dataset_path": dataset.relative_to(ROOT).as_posix(), + "dataset_sha256": file_sha256(dataset), + "algorithm_version": ALGORITHM_VERSION, + "implementation_sha256": implementation_sha256(files), + "files": files, + "historical_frozen_sha256": manifest["frozen_scoring"]["implementation_sha256"], + "evaluator_sha256": file_sha256(Path(__file__)), + "ayanamsa": AYANAMSA, + "node_mode": NODE_MODE, + "candidate_radius_minutes": sorted({case["candidate_radius_minutes"] for case in manifest["cases"]}), + "minute_step": 1, + "release_metrics": manifest["release_metrics"], + "results_previously_seen": True, + "official_valid_independent_blind": False, + "must_not_use_for_tuning": True, + "tie_breaker": manifest["frozen_scoring"]["tie_breaker"], + "metric_rank_definition": "competition_rank_1_plus_strictly_higher_scores_legacy_protocol", + "extra_metric_rank_definition": "score_desc_then_existing_opaque_sha256_total_order", + } + + +def run(freeze_path: Path = FREEZE, dataset: Path = DATASET) -> dict[str, Any]: + frozen = json.loads(freeze_path.read_text(encoding="utf-8")) + actual = freeze_record(dataset) + for key in actual: + if key != "frozen_at_utc" and actual[key] != frozen.get(key): + raise ValueError(f"frozen_record_mismatch:{key}") + manifest = json.loads(dataset.read_text(encoding="utf-8")) + validation = validate(dataset) + invalid = validation["invalid_cases"] + trials = [] + for index, case in enumerate(manifest["cases"], 1): + if case["case_id"] in invalid: + continue + request = _request(case, case["events"]) + candidates = _candidate_moments(case) + facts = build_feature_fact_rows(request, candidates=candidates) + rows, _ = rank_fact_rows(facts, request["events"]) + result = score_fact_ranker_v4(facts, request["events"]) + predicted = _opaque_winner(manifest["benchmark_id"], case["case_id"], rows) + ordered = opaque_order(manifest["benchmark_id"], case["case_id"], rows) + sparse_request = _request(case, case["events"][:1]) + sparse_facts = build_feature_fact_rows(sparse_request, candidates=candidates) + sparse_result = score_fact_ranker_v4(sparse_facts, sparse_request["events"]) + # Truth is revealed only after both full and sparse ranking/decisions. + truth = case["birth"]["time"] + truth_score = next(row["score"] for row in rows if row["time"] == truth) + would_confirm = _would_confirm(result) + trials.append({ + "case_ordinal": index, + "candidate_count": len(rows), + "event_count": len(request["events"]), + "true_rank": 1 + sum(row["score"] > truth_score for row in rows), + "opaque_true_rank": next(i for i, row in enumerate(ordered, 1) if row["time"] == truth), + "minute_error": _clock_distance(predicted, truth), + "would_confirm": would_confirm, + "false_confirmation": would_confirm and predicted != truth, + "insufficient_evidence_rejected": not _would_confirm(sparse_result), + "full_trial_reasons": result["reasons"], + "sparse_trial_reasons": sparse_result["reasons"], + }) + aggregate = summarize_trials(trials, manifest["release_metrics"]) + count = len(trials) + return { + "scope": "fixed_protocol_previously_exposed_v3_rerun", + "evaluated_on": datetime.now(timezone.utc).date().isoformat(), + "frozen_record": frozen, + "implementation_hash_matches_at_replay": True, + "dataset_hash_matches_at_replay": True, + "source_audit_status": manifest["source_audit_status"], + "validation_status": validation["status"], + "valid_public_aa_cases": validation["valid_public_aa_cases"], + "excluded_cases": invalid, + "trial_count": count, + "trials": trials, + **aggregate, + "opaque_exact_top_1_rate": sum(row["opaque_true_rank"] == 1 for row in trials) / count if count else None, + "opaque_exact_top_3_rate": sum(row["opaque_true_rank"] <= 3 for row in trials) / count if count else None, + "official_valid_independent_blind": False, + "official_blind_trial_count": 0, + "is_blind_evaluation": False, + "truth_hidden_from_ranker": True, + "results_previously_seen": True, + "verified_minute_claim_allowed": False, + "status": "blocked_independent_blind_evidence", + "boundary": "Three-event low-information protocol, not a mathematical accuracy lower bound and not representative of real sessions. Historical v3/v4 exposure cannot be undone by freezing today's scorer. No tuning or release claims.", + } + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--freeze", action="store_true", help="Create a new record before replay; never overwrite an existing record") + parser.add_argument("--freeze-path", type=Path, default=FREEZE) + parser.add_argument("--json", action="store_true") + args = parser.parse_args() + if args.freeze: + with args.freeze_path.open("x", encoding="utf-8", newline="\n") as handle: + json.dump(freeze_record(), handle, ensure_ascii=False, indent=2) + handle.write("\n") + print(args.freeze_path.relative_to(ROOT).as_posix()) + else: + print(json.dumps(run(args.freeze_path), ensure_ascii=False, indent=2)) diff --git a/tests/test_rectification_confirmation_and.py b/tests/test_rectification_confirmation_and.py index 9f89ab25..f538fa5e 100644 --- a/tests/test_rectification_confirmation_and.py +++ b/tests/test_rectification_confirmation_and.py @@ -93,6 +93,7 @@ def test_sealed_holdout_contract_matches_v3_report_and_stays_closed() -> None: sidecar = diagnostic["current_tree_unfrozen_diagnostic"] produced_by = product["metrics_produced_by"] current_tree = product["current_tree_scorer"] + rerun = json.loads((ROOT / current_tree["source_report"]).read_text(encoding="utf-8")) unfrozen = product["current_tree_unfrozen_diagnostic"] assert holdout["sealed_benchmark_id"] == report["benchmark_id"] == product["sealed_benchmark_id"] assert holdout["valid_public_aa_cases"] == report["validation"]["valid_public_aa_cases"] @@ -115,7 +116,10 @@ def test_sealed_holdout_contract_matches_v3_report_and_stays_closed() -> None: assert produced_by["algorithm_version"] == report["frozen_scoring"]["algorithm_version"] assert produced_by["implementation_hash_matches_at_replay"] is True assert produced_by["source_report"] == "references/real_case_calibration/minute_rectification_holdout_v3_report.json" - assert current_tree["implementation_sha256"] == diagnostic["frozen_scoring"]["actual_sha256"] + assert current_tree["implementation_sha256"] == rerun["frozen_record"]["implementation_sha256"] + assert current_tree["fixed_protocol_rerun_hash_matches"] is rerun["implementation_hash_matches_at_replay"] is True + assert current_tree["fixed_protocol_rerun_trial_count"] == rerun["trial_count"] == 20 + assert rerun["official_valid_independent_blind"] is False assert current_tree["matches_metrics_scorer"] is False assert current_tree["official_eval_implementation_hash_matches"] is False assert current_tree["official_eval_trial_count"] == diagnostic["trial_count"] == 0 diff --git a/tests/test_rectification_validation_integrity_gate.py b/tests/test_rectification_validation_integrity_gate.py new file mode 100644 index 00000000..dbd2ddfe --- /dev/null +++ b/tests/test_rectification_validation_integrity_gate.py @@ -0,0 +1,16 @@ +"""Collect research integrity regressions in the existing rectification quick gate.""" + +from tests.test_reported_offset_research import ( # noqa: F401 + test_cross_midnight_real_engine_scores_match_candidate_date_replay, + test_offset_beyond_radius_has_no_truth_candidate, + test_opaque_tie_break_is_independent_of_truth_and_input_order, + test_recorded_specification_and_all_prespecified_cells, + test_shifted_window_preserves_dates_across_midnight, + test_zero_offset_centres_on_truth_and_has_complete_grid, +) +from tests.test_sealed_holdout_contract_freshness import ( # noqa: F401 + test_changed_frozen_identity_fails_before_any_replay, + test_contract_tracks_actual_current_scorer_and_dataset_audit, + test_fixed_protocol_rerun_is_auditable_but_never_independent_blind, + test_frozen_record_matches_dataset_scorer_and_evaluator_bytes, +) diff --git a/tests/test_reported_offset_research.py b/tests/test_reported_offset_research.py new file mode 100644 index 00000000..2814ca9e --- /dev/null +++ b/tests/test_reported_offset_research.py @@ -0,0 +1,117 @@ +from __future__ import annotations + +import json +from datetime import datetime, timedelta + +import pytest + +from scripts.research import reported_offset_sweep as sweep +from scripts.research.sealed_holdout_rerun import DATASET, opaque_order + + +def public_case(): + return json.loads(DATASET.read_text(encoding="utf-8"))["cases"][0] + + +def test_zero_offset_centres_on_truth_and_has_complete_grid(): + case = public_case() + truth = datetime.fromisoformat(f"{case['birth']['date']}T{case['birth']['time']}:00") + request, candidates = sweep.shifted_window(case, 0, 15) + assert candidates[15] == truth + assert len(candidates) == 31 + assert request["start_time"] == candidates[0].strftime("%H:%M") + assert request["end_time"] == candidates[-1].strftime("%H:%M") + assert truth in candidates + assert not any("truth" in key or "offset" in key for key in request) + + +@pytest.mark.parametrize("offset", [-30, -20, 20, 30]) +def test_offset_beyond_radius_has_no_truth_candidate(offset): + case = public_case() + truth = datetime.fromisoformat(f"{case['birth']['date']}T{case['birth']['time']}:00") + _, candidates = sweep.shifted_window(case, offset, 15) + assert truth not in candidates + rows = [{"time": moment.strftime("%H:%M"), "score": 1} for moment in candidates] + metrics = sweep.reveal_metrics(rows, candidates, candidates, truth, "test", "test") + assert metrics["truth_in_window"] is False + assert metrics["true_rank"] is None + assert metrics["delivery_covers_truth"] is False + + +def test_opaque_tie_break_is_independent_of_truth_and_input_order(): + # Explicitly synthetic unit inputs, not a fabricated engine-contract fixture. + moments = [datetime(2000, 1, 1, 12) + timedelta(minutes=i) for i in range(3)] + rows = [{"time": moment.strftime("%H:%M"), "score": 1} for moment in moments] + ordered = opaque_order("test", "synthetic", rows) + assert ordered == opaque_order("test", "synthetic", list(reversed(rows))) + winner = next(moment for moment in moments if moment.strftime("%H:%M") == ordered[0]["time"]) + for truth in moments: + metrics = sweep.reveal_metrics(rows, moments, moments, truth, "test", "synthetic") + assert metrics["top_1_hit"] is (truth == winner) + assert metrics["true_rank"] == 1 + next(i for i, row in enumerate(ordered) if row["time"] == truth.strftime("%H:%M")) + + +@pytest.mark.parametrize("clock,offset", [("00:02", -5), ("23:58", 5)]) +def test_shifted_window_preserves_dates_across_midnight(clock, offset): + case = public_case() + # Synthetic boundary mutation of the public case; no real user's birth data. + case["birth"] = {**case["birth"], "time": clock} + request, candidates = sweep.shifted_window(case, offset, 15) + truth = datetime.fromisoformat(f"{case['birth']['date']}T{clock}:00") + assert candidates[15] == truth + timedelta(minutes=offset) + assert request["birth_date"] == candidates[0].date().isoformat() + rows = [{"time": moment.strftime("%H:%M"), "score": int(moment == truth)} for moment in candidates] + metrics = sweep.reveal_metrics(rows, candidates, candidates, truth, "test", "synthetic") + assert metrics["top_1_minute_error"] == 0 + assert metrics["truth_in_window"] is True + assert metrics["delivery_covers_truth"] is True + assert metrics["delivery_width_minutes"] == 31 + + +def test_cross_midnight_real_engine_scores_match_candidate_date_replay(): + case = json.loads(DATASET.read_text(encoding="utf-8"))["cases"][5] + request, candidates = sweep.shifted_window(case, 0, 60) + assert len({candidate.date() for candidate in candidates}) == 2 + contexts = sweep.compute_candidate_static_contexts(request, candidates=candidates) + grouped = sweep.score_window(request, contexts) + expected = [] + for context in contexts: + dated = {**request, "birth_date": context["candidate_at"].date().isoformat()} + built = sweep.build_event_contribution_matrix(dated, static_contexts=[context]) + expected.extend(sweep.score_from_matrix(dated, built)) + assert grouped == expected + old_matrix = sweep.build_event_contribution_matrix(request, static_contexts=contexts) + old_rows = sweep.score_from_matrix(request, old_matrix) + assert old_rows != expected + + +def test_recorded_specification_and_all_prespecified_cells(): + report = json.loads((sweep.ROOT / "docs/research/reported_offset_2026_09_20.json").read_text(encoding="utf-8")) + spec = report["specification"] + assert spec["ayanamsa"] == "raman" + assert spec["node_mode"] == "mean" + assert spec["radii_minutes"] == list(sweep.RADII) + assert spec["offsets_minutes"] == list(sweep.OFFSETS) + assert spec["minute_step"] == 1 + assert spec["implementation_sha256_prefix"] == spec["implementation_sha256"][:16] + assert spec["dataset_sha256"] == sweep.file_sha256(DATASET) + assert spec["evaluator_sha256"] == sweep.file_sha256(sweep.ROOT / "scripts/research/reported_offset_sweep.py") + assert spec["production_scoring_sha256"] == sweep.implementation_sha256(spec["production_scoring_files"]) + assert spec["research_implementation_sha256"] == sweep.implementation_sha256(spec["research_files"]) + assert spec["replay_revision"] == "candidate_date_grouped_v2" + assert spec["truth_hidden_from_ranker"] is True + assert spec["is_blind_evaluation"] is False + assert report["trial_count"] == 20 * len(sweep.RADII) * len(sweep.OFFSETS) + assert len(report["summary"]) == len(sweep.RADII) * len(sweep.OFFSETS) + assert report["summary"] == sweep.summarize(report["trials"], sweep.RADII, sweep.OFFSETS) + assert {(row["case_ordinal"], row["radius_minutes"], row["offset_minutes"]) for row in report["trials"]} == { + (case, radius, offset) for case in range(1, 21) for radius in sweep.RADII for offset in sweep.OFFSETS + } + assert report["widening_geometry"]["radius_120_scored"] is False + for trial in report["trials"]: + assert not ({"predicted_time", "birth", "latitude", "longitude"} & trial.keys()) + for row in report["summary"]: + expected = float(abs(row["offset_minutes"]) <= row["radius_minutes"]) + assert row["truth_in_window_rate"] == expected + assert row["delivery_coverage_rate"] <= expected + assert row["top_1_rate"] <= expected diff --git a/tests/test_sealed_holdout_contract_freshness.py b/tests/test_sealed_holdout_contract_freshness.py new file mode 100644 index 00000000..8069e30f --- /dev/null +++ b/tests/test_sealed_holdout_contract_freshness.py @@ -0,0 +1,81 @@ +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from scripts.minute_rectification_blind_eval import implementation_sha256, summarize_trials +from scripts.rectification.sealed_holdout import holdout_passed, load_sealed_minute_holdout +from scripts.research.sealed_holdout_rerun import DATASET, FREEZE, REPORT, file_sha256, freeze_record, run + +ROOT = Path(__file__).resolve().parents[1] + + +def read(path): + return json.loads(path.read_text(encoding="utf-8")) + + +def test_contract_tracks_actual_current_scorer_and_dataset_audit(): + dataset = read(DATASET) + contract = read(ROOT / "references/rectification_sealed_holdout.v1.json") + actual_hash = implementation_sha256(dataset["frozen_scoring"]["files"]) + assert contract["current_tree_scorer"]["implementation_sha256"] == actual_hash + assert contract["source_audit_status"] == dataset["source_audit_status"] + assert contract["evaluated_on"] == read(REPORT)["evaluated_on"] + assert contract["status"] == "not_ready" + assert contract["confirmation_coverage_rate"] == 0.0 + assert holdout_passed(load_sealed_minute_holdout()) is False + + +def test_frozen_record_matches_dataset_scorer_and_evaluator_bytes(): + frozen = read(FREEZE) + actual = freeze_record() + assert {key: value for key, value in frozen.items() if key != "frozen_at_utc"} == { + key: value for key, value in actual.items() if key != "frozen_at_utc" + } + assert frozen["dataset_sha256"] == file_sha256(DATASET) + assert len(frozen["files"]) == 12 + assert read(DATASET)["frozen_scoring"]["implementation_sha256"] == frozen["historical_frozen_sha256"] + + +def test_fixed_protocol_rerun_is_auditable_but_never_independent_blind(): + report = read(REPORT) + contract = read(ROOT / "references/rectification_sealed_holdout.v1.json") + scorer = contract["current_tree_scorer"] + rerun = contract["current_tree_fixed_protocol_rerun"] + assert report["frozen_record"] == read(FREEZE) + assert report["trial_count"] == len(report["trials"]) == 20 + assert report["excluded_cases"] == [] + aggregate = summarize_trials(report["trials"], read(DATASET)["release_metrics"]) + assert report["metrics"] == aggregate["metrics"] == scorer["metrics"] + assert report["metric_gates_passed"] == aggregate["metric_gates_passed"] + assert scorer["source_report"] == REPORT.relative_to(ROOT).as_posix() + assert scorer["implementation_sha256"] == report["frozen_record"]["implementation_sha256"] + assert scorer["fixed_protocol_rerun_trial_count"] == report["trial_count"] + assert scorer["fixed_protocol_rerun_hash_matches"] is True + assert scorer["official_eval_trial_count"] == report["official_blind_trial_count"] == 0 + assert scorer["official_eval_implementation_hash_matches"] is False + for item in (rerun, report): + assert item["official_valid_independent_blind"] is False + assert item["is_blind_evaluation"] is False + assert item["truth_hidden_from_ranker"] is True + assert item["results_previously_seen"] is True + assert item["verified_minute_claim_allowed"] is False + assert rerun["must_not_claim_as_release_metrics"] is True + assert all(row["event_count"] == 3 for row in report["trials"]) + # Never persist actual candidate/truth times or coordinates in report artifacts. + for row in report["trials"]: + assert not ({"predicted_time", "published_truth_revealed_after_ranking", "latitude", "longitude"} & row.keys()) + + +def test_changed_frozen_identity_fails_before_any_replay(tmp_path, monkeypatch): + frozen = read(FREEZE) + frozen["implementation_sha256"] = "0" * 64 + path = tmp_path / "bad-freeze.json" + path.write_text(json.dumps(frozen), encoding="utf-8") + def unexpected(*args, **kwargs): + raise AssertionError("must reject identity before scoring") + monkeypatch.setattr("scripts.research.sealed_holdout_rerun.build_feature_fact_rows", unexpected) + with pytest.raises(ValueError, match="frozen_record_mismatch:implementation_sha256"): + run(path)