research(rectification): holdout v5 checkpoint — protocol, roster (47 new AA cases), quoted events, build script

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017eEAG8HD3mm8gsKXgk8uU8

docs/research/holdout_v5_protocol_2026_09_29.md

@@ -0,0 +1,54 @@
+# Holdout v5 选例与核对协议（2026-09-29）
+
+任务书：`docs/tasks/TASK-rectification-holdout-expansion-20260929.md`。本页写清 v5 的入选标准、分层配额、排除规则与来源记录格式；数字与统计见 `holdout_v5_build_2026_09_29.md`。
+
+## 1. 入选标准
+
+1. **出生资料**：Astro-Databank 记录、Rodden **AA**（出生证 / 出生记录 / 医院记录）。A、B、C、DD、X、XX 一律不收。
+2. **公开名人**：只收有英文 Wikipedia 条目的公众人物；不收本仓任何真实用户、库主、协作者或其亲友。
+3. **事件**：每例 ≥7 件带年月事件、≥4 个领域；领域键取 `scripts/active_rectification_event_engine.py::DOMAIN_CONFIG`。
+4. **来源独立**：事件出处与出生资料出处互不相同（`independent_of_birth_source=true`）。
+5. **精度诚实**：`year` / `month` / `day` 按来源页所写的精度标注；来源只写"某年"就标 `year`，不得补成 1 月或 1 日。
+
+## 2. 排除规则
+
+- 出生时间在 Astro-Databank 上标为 rectified / from memory / 无时间（X）的不收。
+- 同一家族多人只取一人（v4 已有 Sean Lennon、Joseph Kennedy III；v5 新增的 Serena 与 Venus Williams 为姐妹，按本条**仍收**——两人出生资料独立、事件独立，且任务书分层需要；已在构建说明中注明）。
+- 核对未通过（时区不一致、坐标与地名不符）的记录**不收**，不改数字凑合，见 §4。
+
+## 3. 分层配额（任务书硬红线 4）
+
+| 维度 | 配额 |
+| --- | --- |
+| 出生年代 | ≥3 个年代段 |
+| 纬度带 | \|φ\| <25° / 25–45° / >45° 三档都有 |
+| 南半球 | ≥8 例 |
+| 出生时间 22:00–02:00 | ≥6 例 |
+| UTC+8 | ≥6 例 |
+
+达不到的写进 PROGRESS，不得静默。
+
+## 4. 来源核对流程（每例）
+
+1. **评级与出生资料**：astro.com 在 2026-09-29 对自动访问返回"One moment..."拦截页，Wayback 快照间歇不可达。因此评级与出生资料从 Astro-Seek 的 famous-people 页读取——该页逐字转载 Astro-Databank 的评级字母、采集人（Rodden / Scholfield / Borges …）、出生日期、当地时间、地点、坐标与 UTC 修正。记录里同时写 Astro-Databank 原页 URL（`birth.source.url`）与镜像页 URL（`birth.source.verified_via.url`）。**这是环境缺口**：验收时若 astro.com 可访问，应抽样回到原页复核。
+2. **时区核对**：用 `timezonefinder` + `zoneinfo` 按坐标与出生时刻算 UTC 偏移，与记录偏移比对。三种通过规则：
+   - `zoneinfo_match`：完全一致；
+   - `standard_time_kept_dst_disagreement`：tzdata 把时区代表城市的夏令时史套到整个时区，而出生记录保持标准时（例：1941 年 Duluth、1922 年 Grand Rapids、1963 年 Owensboro）；
+   - `lmt_from_birthplace_longitude`：1890 年前后欧洲的地方平时，tzdata 只有代表城市的 LMT，记录用出生地经度的 LMT（例：1879 Ulm、1890 Lille）。
+   其他不一致一律拒收（本轮拒收 Borges 1899 Buenos Aires、Allende 1908 Valparaíso）。
+3. **地名核对**：Nominatim 地理编码记录里的地名，与记录坐标距离 ≤60 km。不一致拒收（本轮拒收 Corazon Aquino：记录地名 Paniqui，镜像坐标为马尼拉，相距 129 km）。
+4. **事件**：从英文 Wikipedia 正文（API `extracts`，纯文本）逐句筛出带年份的候选句，人工选择并标注领域 / 日期 / 精度；每条事件保存来源页 URL 与**逐字引文**（`source.quote`），构建脚本核对引文确实出现在页面文本里，找不到即拒收。模型草稿（候选句列表）不进数据集。
+
+## 5. 数据文件
+
+| 文件 | 内容 |
+| --- | --- |
+| `references/real_case_calibration/holdout_v5_roster.json` | 新增例的出生记录、评级、核对结果（`checks`） |
+| `references/real_case_calibration/minute_rectification_holdout_v5_events.json` | 新增例的事件（含引文） |
+| `references/real_case_calibration/minute_rectification_holdout_v5.json` | 构建产物：v4 的 20 例原样 + 新增例 |
+| `scripts/research/holdout_v5_build.py` | 构建脚本（`--check` 只校验不写） |
+| `tests/test_holdout_v5_schema.py` | schema 测试 |
+
+## 6. 口径
+
+ayanamsa `raman`、node mode `mean`、候选步长 2 分钟、三档半径 ±10 / ±30 / ±60 写在数据集上。v5 是开放评价集（`truth_hidden_from_ranker=false`），不是封存盲测；`sealed_holdout_rerun.py` 与 reported-offset 记录不因本单改变。

scripts/research/holdout_v5_build.py

@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+"""Build minute_rectification_holdout_v5 from the v4 public AA set plus the reviewed v5 roster.
+
+Inputs (all under references/real_case_calibration/):
+  - minute_rectification_holdout_v4.json      the 20 inherited cases, copied verbatim
+  - holdout_v5_roster.json                    new public Rodden-AA birth records + checks
+  - minute_rectification_holdout_v5_events.json  reviewed, quoted events per new case
+
+Output: minute_rectification_holdout_v5.json (deterministic; two builds are byte-identical).
+Public biographies only. No private user data. No scoring code is touched.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from collections import Counter
+from pathlib import Path
+from typing import Any
+
+ROOT = Path(__file__).resolve().parents[2]
+CAL = ROOT / "references" / "real_case_calibration"
+V4_PATH = CAL / "minute_rectification_holdout_v4.json"
+ROSTER_PATH = CAL / "holdout_v5_roster.json"
+EVENTS_PATH = CAL / "minute_rectification_holdout_v5_events.json"
+V5_PATH = CAL / "minute_rectification_holdout_v5.json"
+
+RADII = (10, 30, 60)
+FALSE_MINUTE_OFFSETS = [-10, -5, -2, -1, 1, 2, 5, 10]
+MIN_EVENTS = 7
+MIN_DOMAINS = 4
+MIN_CASES = 60
+DOMAINS = {"education", "relocation", "relationship", "career", "finance", "health_pressure", "family", "appearance", "occupation"}
+PRECISION_FORMAT = {"year": 4, "month": 7, "day": 10}
+
+
+def load(path: Path) -> Any:
+    return json.loads(path.read_text(encoding="utf-8"))
+
+
+def latitude_band(lat: float) -> str:
+    a = abs(lat)
+    if a < 25:
+        return "lt25"
+    if a <= 45:
+        return "25to45"
+    return "gt45"
+
+
+def night_birth(time: str) -> bool:
+    hour = int(time.split(":")[0])
+    return hour >= 22 or hour < 2
+
+
+def strata_for(birth: dict[str, Any]) -> dict[str, Any]:
+    year = int(birth["date"][:4])
+    return {
+        "decade": f"{year // 10 * 10}s",
+        "latitude_band": latitude_band(float(birth["latitude"])),
+        "hemisphere": "south" if float(birth["latitude"]) < 0 else "north",
+        "night_birth_22_to_02": night_birth(birth["time"]),
+        "utc_plus_8": abs(float(birth["timezone_offset"]) - 8.0) < 0.01,
+    }
+
+
+def validate_event(case_id: str, event: dict[str, Any], problems: list[str]) -> None:
+    if event.get("domain") not in DOMAINS:
+        problems.append(f"{case_id}: bad domain {event.get('domain')}")
+    precision = event.get("precision")
+    date = str(event.get("date", ""))
+    if precision not in PRECISION_FORMAT or len(date) != PRECISION_FORMAT[precision]:
+        problems.append(f"{case_id}: date {date!r} does not match precision {precision!r}")
+    source = event.get("source") or {}
+    if not source.get("url") or not source.get("independent_of_birth_source"):
+        problems.append(f"{case_id}: event {event.get('id')} lacks an independent source url")
+
+
+def build_new_case(roster_row: dict[str, Any], events: list[dict[str, Any]], problems: list[str]) -> dict[str, Any]:
+    case_id = roster_row["case_id"]
+    birth = roster_row["birth"]
+    if (birth.get("source") or {}).get("rodden_rating") != "AA":
+        problems.append(f"{case_id}: rodden_rating is not AA")
+    checks = roster_row.get("checks") or {}
+    if not checks.get("timezone_agrees"):
+        problems.append(f"{case_id}: timezone check did not agree")
+    if checks.get("place_distance_km") is None or float(checks["place_distance_km"]) > 60:
+        problems.append(f"{case_id}: place/coordinate check missing or too far")
+    for event in events:
+        validate_event(case_id, event, problems)
+    domains = sorted({e["domain"] for e in events})
+    if len(events) < MIN_EVENTS or len(domains) < MIN_DOMAINS:
+        problems.append(f"{case_id}: {len(events)} events / {len(domains)} domains below gate")
+    return {
+        "case_id": case_id,
+        "subject_label": roster_row["subject_label"],
+        "holdout_partition": "open_scale_evaluation",
+        "excluded_from_tuning": True,
+        "inherited_from_v4": False,
+        "candidate_radius_minutes": RADII[0],
+        "false_minute_offsets": list(FALSE_MINUTE_OFFSETS),
+        "birth": birth,
+        "checks": checks,
+        "strata": strata_for(birth),
+        "events": sorted(events, key=lambda e: (e["date"], e["id"])),
+        "candidate_radii_minutes": list(RADII),
+        "true_minute": birth["time"],
+        "true_cluster_note": "Cluster membership depends on the scoring variant and radius; recorded in the sweep results.",
+    }
+
+
+def inherit_v4_case(case: dict[str, Any]) -> dict[str, Any]:
+    out = dict(case)
+    out["inherited_from_v4"] = True
+    out["strata"] = strata_for(case["birth"])
+    return out
+
+
+def summarize(cases: list[dict[str, Any]]) -> dict[str, Any]:
+    strata = [c["strata"] for c in cases]
+    return {
+        "cases": len(cases),
+        "inherited_from_v4": sum(1 for c in cases if c.get("inherited_from_v4")),
+        "events_total": sum(len(c["events"]) for c in cases),
+        "events_by_precision": dict(sorted(Counter(e["precision"] for c in cases for e in c["events"]).items())),
+        "events_by_domain": dict(sorted(Counter(e["domain"] for c in cases for e in c["events"]).items())),
+        "decades": dict(sorted(Counter(s["decade"] for s in strata).items())),
+        "latitude_bands": dict(sorted(Counter(s["latitude_band"] for s in strata).items())),
+        "southern_hemisphere": sum(1 for s in strata if s["hemisphere"] == "south"),
+        "night_birth_22_to_02": sum(1 for s in strata if s["night_birth_22_to_02"]),
+        "utc_plus_8": sum(1 for s in strata if s["utc_plus_8"]),
+        "countries": dict(sorted(Counter(c["birth"]["place"].split(",")[-1].strip() for c in cases).items())),
+    }
+
+
+def build(v4: dict[str, Any], roster: dict[str, Any], events_by_case: dict[str, Any]) -> tuple[dict[str, Any], list[str]]:
+    problems: list[str] = []
+    cases = [inherit_v4_case(c) for c in v4["cases"]]
+    known = {c["case_id"] for c in cases}
+    for row in roster["cases"]:
+        case_id = row["case_id"]
+        if case_id in known:
+            problems.append(f"duplicate case_id {case_id}")
+            continue
+        events = (events_by_case.get(case_id) or {}).get("events") or []
+        if not events:
+            problems.append(f"{case_id}: no events file entry")
+        cases.append(build_new_case(row, events, problems))
+        known.add(case_id)
+    cases.sort(key=lambda c: (not c.get("inherited_from_v4"), c["case_id"]))
+    if len(cases) < MIN_CASES:
+        problems.append(f"only {len(cases)} cases (< {MIN_CASES})")
+    payload = {
+        "schema_version": "minute-rectification-holdout-v5",
+        "benchmark_id": "minute_rectification_scale_holdout_v5",
+        "description": (
+            "Public Rodden-AA minute set: the 20 v4 cases verbatim plus reviewed public biographies, "
+            "each with at least 7 dated events and 4 domains, scored at ±10 / ±30 / ±60. "
+            "Open evaluation for scoring-scale research, not a sealed blind release gate."
+        ),
+        "truth_hidden_from_ranker": False,
+        "frozen_before_replay": False,
+        "source_audit_status": "public_biography_expansion_2026_09_29",
+        "ayanamsa": v4["ayanamsa"],
+        "node_mode": v4["node_mode"],
+        "minute_step": 2,
+        "candidate_radii_minutes": list(RADII),
+        "minimum_gate": {"public_aa_cases": MIN_CASES, "events_per_case": MIN_EVENTS, "domains_per_case": MIN_DOMAINS},
+        "parent_dataset": str(V4_PATH.relative_to(ROOT)).replace("\\", "/"),
+        "roster": str(ROSTER_PATH.relative_to(ROOT)).replace("\\", "/"),
+        "events_source": str(EVENTS_PATH.relative_to(ROOT)).replace("\\", "/"),
+        "birth_record_protocol": roster.get("protocol"),
+        "summary": summarize(cases),
+        "cases": cases,
+        "boundary": (
+            "Not a sealed blind holdout. Events are public biographies quoted from their source pages. "
+            "Do not mix these numbers with upstream true-node runs."
+        ),
+    }
+    return payload, problems
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--check", action="store_true", help="validate only; do not write")
+    args = parser.parse_args()
+    payload, problems = build(load(V4_PATH), load(ROSTER_PATH), load(EVENTS_PATH))
+    for problem in problems:
+        print("PROBLEM:", problem)
+    if problems:
+        return 1
+    if not args.check:
+        V5_PATH.write_text(json.dumps(payload, ensure_ascii=False, indent=1) + "\n", encoding="utf-8")
+        print(f"wrote {V5_PATH.relative_to(ROOT)}")
+    print(json.dumps(payload["summary"], ensure_ascii=False, indent=1))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())