docs(research): holdout v5 full-set baseline scorecard and progress close-out (BUG-1090)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017eEAG8HD3mm8gsKXgk8uU8
This commit is contained in:
Jesse_Chen
2026-09-29 19:52:43 +08:00
co-authored by Claude Fable 5.1
parent e380026641
commit 88dca263fe
3 changed files with 274 additions and 3 deletions
@@ -0,0 +1,244 @@
{
"dataset": "references/real_case_calibration/minute_rectification_holdout_v5.json",
"case_filter": null,
"radii": [
10,
30,
60
],
"variant": "baseline (production scoring, no research variants)",
"ayanamsa": "raman",
"node_mode": "mean",
"sweep": {
"baseline": {
"10": {
"n": 77,
"top1": 0.1818,
"coverage": 1.0,
"width_median": 21,
"tie": 0.026,
"entropy0": 2.6239,
"entropy6": 2.158,
"squeezed": 0,
"too_narrow": 0,
"replay_top1": 0.6364
},
"30": {
"n": 77,
"top1": 0.0779,
"coverage": 1.0,
"width_median": 61,
"tie": 0.026,
"entropy0": 4.1018,
"entropy6": 2.5422,
"squeezed": 0,
"too_narrow": 0,
"replay_top1": 0.4935
},
"60": {
"n": 77,
"top1": 0.0649,
"coverage": 1.0,
"width_median": 121,
"tie": 0.013,
"entropy0": 5.0533,
"entropy6": 2.6466,
"squeezed": 0,
"too_narrow": 0,
"replay_top1": 0.2597
}
},
"case_count": 77,
"errors": []
},
"cluster_width": {
"replay": {
"10": {
"n": 77,
"top1": 0.6753,
"coverage": 0.987,
"width_median": 15,
"tie": 0.0,
"entropy0": 2.624,
"entropy6": 2.0587,
"squeezed": 1,
"independent": 1.0,
"public_median": 6,
"merge_median": 0,
"raw_median": 6,
"elim_median": 1,
"union_alive_median": 17,
"engine_union_median": 21
},
"30": {
"n": 77,
"top1": 0.4935,
"coverage": 0.987,
"width_median": 35,
"tie": 0.0,
"entropy0": 4.1023,
"entropy6": 3.3001,
"squeezed": 1,
"independent": 1.0,
"public_median": 17,
"merge_median": 0,
"raw_median": 17,
"elim_median": 7,
"union_alive_median": 43,
"engine_union_median": 61
},
"60": {
"n": 77,
"top1": 0.3247,
"coverage": 0.974,
"width_median": 63,
"tie": 0.0,
"entropy0": 5.0543,
"entropy6": 4.1692,
"squeezed": 2,
"independent": 1.0,
"public_median": 32,
"merge_median": 0,
"raw_median": 32,
"elim_median": 13,
"union_alive_median": 101,
"engine_union_median": 121
}
},
"case_count": 77,
"errors": []
},
"futile_collect": {
"summaries": [
{
"radius": 10,
"direction": "truth",
"n": 77,
"baseline": {
"truth_in_range": 76,
"truth_in_range_rate": 0.987,
"median_width": 13,
"mean_injected": 1.82
},
"d1": {
"truth_in_range": 76,
"truth_in_range_rate": 0.987,
"median_width": 15,
"mean_injected": 0.0
},
"width_worse_cases": 6,
"truth_lost_cases": 0,
"gate_pass": false,
"errors": 0
},
{
"radius": 10,
"direction": "opposite",
"n": 77,
"baseline": {
"truth_in_range": 76,
"truth_in_range_rate": 0.987,
"median_width": 13,
"mean_injected": 1.82
},
"d1": {
"truth_in_range": 76,
"truth_in_range_rate": 0.987,
"median_width": 15,
"mean_injected": 0.0
},
"width_worse_cases": 4,
"truth_lost_cases": 0,
"gate_pass": false,
"errors": 0
},
{
"radius": 30,
"direction": "truth",
"n": 77,
"baseline": {
"truth_in_range": 75,
"truth_in_range_rate": 0.974,
"median_width": 35,
"mean_injected": 1.82
},
"d1": {
"truth_in_range": 76,
"truth_in_range_rate": 0.987,
"median_width": 35,
"mean_injected": 0.0
},
"width_worse_cases": 9,
"truth_lost_cases": 0,
"gate_pass": true,
"errors": 0
},
{
"radius": 30,
"direction": "opposite",
"n": 77,
"baseline": {
"truth_in_range": 75,
"truth_in_range_rate": 0.974,
"median_width": 35,
"mean_injected": 1.82
},
"d1": {
"truth_in_range": 76,
"truth_in_range_rate": 0.987,
"median_width": 35,
"mean_injected": 0.0
},
"width_worse_cases": 10,
"truth_lost_cases": 0,
"gate_pass": true,
"errors": 0
},
{
"radius": 60,
"direction": "truth",
"n": 77,
"baseline": {
"truth_in_range": 75,
"truth_in_range_rate": 0.974,
"median_width": 61,
"mean_injected": 1.82
},
"d1": {
"truth_in_range": 75,
"truth_in_range_rate": 0.974,
"median_width": 63,
"mean_injected": 0.0
},
"width_worse_cases": 15,
"truth_lost_cases": 0,
"gate_pass": false,
"errors": 0
},
{
"radius": 60,
"direction": "opposite",
"n": 77,
"baseline": {
"truth_in_range": 75,
"truth_in_range_rate": 0.974,
"median_width": 61,
"mean_injected": 1.82
},
"d1": {
"truth_in_range": 75,
"truth_in_range_rate": 0.974,
"median_width": 63,
"mean_injected": 0.0
},
"width_worse_cases": 13,
"truth_lost_cases": 0,
"gate_pass": false,
"errors": 0
}
],
"errors": [],
"case_count": 77
},
"elapsed_s": 1562.7
}
+17 -2
View File
@@ -33,7 +33,7 @@ UTC+8 缺口说明:Astro-Seek 镜像上菲律宾 126 例、中国 205 例、
## 2. 出生资料核对结果
- 评级:57 例新增全部 AA(镜像页转载的 Astro-Databank 评级字母;astro.com 原站 2026-09-29 拦截自动访问,见协议 §4.1 的环境缺口)。
- 时区:`zoneinfo_match` 51、`lmt_from_birthplace_longitude` 3(1853 Zundert、1859 Edinburgh、1879 Ulm)、`standard_time_kept_dst_disagreement` 3(1922 Grand Rapids、1941 Duluth、1963 Owensboro)。
- 时区:`zoneinfo_match` 51、`lmt_from_birthplace_longitude` 3(1879 Ulm、1890 Lille、1902 Diamantina:tzdata 只有时区代表城市的 LMT,记录用出生地经度的 LMT)、`standard_time_kept_dst_disagreement` 3(1922 Grand Rapids、1941 Duluth、1963 Owensboro:tzdata 把代表城市的夏令时史套到整个时区,记录保持标准时)。1859 Edinburgh 按 tzdata 自 1847 起为 GMT,记录 0:00 与之一致。
- 拒收(不改数字凑合):Jorge Luis Borges 1899(记录 −3.9h,zoneinfo −4.28h)、Salvador Allende 1908 与 Claudio Arrau 1903(记录 −4.78 / −4.80h,zoneinfo −4.71h,智利 1910 年前 LMT 与记录不一致)、Corazon Aquino(地名核对)、Juan Perón(Wikipedia 生日 10 月 8 日与记录 10 月 7 日不一致)、Pope Francis(只凑到 3 个领域)。
- 每条事件带来源页逐字引文,构建脚本核对引文存在于 Wikipedia 纯文本后才写入。
@@ -51,7 +51,22 @@ UTC+8 缺口说明:Astro-Seek 镜像上菲律宾 126 例、中国 205 例、
## 4. v5 全集基线成绩单
<!-- V5_BASELINE_TABLE -->
结果 JSON:`docs/research/holdout_v5_baseline_v5_all_2026_09_29.json`(77 例,`baseline` 变体,耗时 1563 s)。括号内是 v4 的 20 例(09-14 已发布)作对照;**v5 数字只是成绩单,不是对任何方案的判定**。
| 脚本 | 指标 | ±10 | ±30 | ±60 |
| --- | --- | --- | --- | --- |
| `minute_resolution_sweep`(引擎先验) | 头名命中 | 0.18(0.35) | 0.08(0.15) | 0.06(0.10) |
| | 区间覆盖 / 宽度中位 / 并列率 | 1.00 / 21 / 0.03 | 1.00 / 61 / 0.03 | 1.00 / 121 / 0.01 |
| | 六题回放头名 | 0.64(0.80) | 0.49(0.55) | 0.26(0.35) |
| `cluster_width_probe`(六题后线上口径) | 头名簇命中 | 0.68(0.80) | 0.49(0.55) | 0.32(0.40) |
| | 真值在区间 | 76/77(20/20) | 76/77(20/20) | 75/77(20/20) |
| | 宽度中位 | 15(15) | 35(33) | 63(56) |
| `futile_collect_stop_replay`(D1 口径,truth / opposite) | 真值在区间 | 76/77,76/77 | 76/77,76/77 | 75/77,75/77 |
| | 宽度中位 | 15,15 | 35,35 | 63,63 |
真值被挤出区间的例(`cluster_width_probe` 六题回放):Arthur Conan Doyle 1859(±10、±30,答 0 题即被引擎先验排除)、Charles de Gaulle 1890(±60,答 0 题)、Manuel Puig 1932(±60,答 6 题后)。前两例都是 LMT 时代的记录(1859 Edinburgh 按 tzdata 记 GMT 0:00,1890 Lille 记出生地 LMT +0:12),出生分钟本身可能就有十来分钟的历史时区误差——这是 v5 交给研究单的一个已知风险,不是本单能裁决的。
读法:v5 上引擎先验头名从 0.35 掉到 0.18、六题回放从 0.80 掉到 0.64、覆盖从 20/20 变成 76/77——v4 的 20 例是偏乐观的样本;后续 R-A / R-B / R-C 的基线以本表为准。
## 5. 与 v4 的差异
@@ -40,7 +40,13 @@ v5 已建:**77 例**(v4 20 例原样继承 + 57 例新增公开 Rodden-AA)
- **硬红线 5**:`--dataset v4` 复跑,三个脚本共 7+12+6 组数字与 `minute_resolution_2026_09_14.json`、`cluster_width_2026_09_14.json`、`futile_collect_stop_replay_2026_09_29.json` 逐格一致(表在 `holdout_v5_build_2026_09_29.md` §3;JSON `holdout_v5_baseline_v4_all_2026_09_29.json`)。
- v5 全集:`holdout_v5_baseline_v5_all_2026_09_29.json`,数字见构建说明 §4。
<!-- V5_PROGRESS_TABLE -->
| 半径 | 引擎先验头名(v4) | 六题回放头名(v4) | 六题后真值在区间(v4) | 六题后宽度中位(v4) | D1 六格真值在区间 truth / opposite |
| --- | --- | --- | --- | --- | --- |
| ±10 | 0.18(0.35) | 0.64(0.80) | 76/77(20/20) | 15(15) | 76/77,76/77 |
| ±30 | 0.08(0.15) | 0.49(0.55) | 76/77(20/20) | 35(33) | 76/77,76/77 |
| ±60 | 0.06(0.10) | 0.26(0.35) | 75/77(20/20) | 63(56) | 75/77,75/77 |
真值被挤出的例与说明见构建说明 §4。
## T5 文档
@@ -56,3 +62,9 @@ v5 已建:**77 例**(v4 20 例原样继承 + 57 例新增公开 Rodden-AA)
- UTC+8 差 1 例;astro.com 原站抽样复核;Rod Chang(台湾 AA)若能找到可靠传记可补。
- 拒收的 3 例时区不一致(阿根廷 / 智利 1899–1908)值得在 astro.com 可访问后核对原记录的时区注释。
## 快速门
- 开工基线(`origin/staging` @ `5febb111`,本 worktree,Node 22 in PATH):Python 步 **1001 passed / 1 skipped**;随后 `npm test` 在 `frontend` 报 `sh: 1: tsx: not found`(worktree 没装前端依赖)→ 门失败。
- 改动后(`ca54d357`):Python 步 **1001 passed / 1 skipped**,同一处 `tsx: not found`,与基线逐条一致。前端零改动,未在 worktree 安装依赖(环境缺口,非回归)。
- 定向:`tests/test_holdout_v5_schema.py` 9 passed;`test_minute_resolution_research.py` + `test_cluster_width_research.py` + `test_repo_privacy_markers.py` 全绿。