research(rectification): baseline runner never rewrites the published 09-14 report

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017eEAG8HD3mm8gsKXgk8uU8
This commit is contained in:
Jesse_Chen
2026-09-29 19:52:43 +08:00
co-authored by Claude Fable 5.1
parent 88dca263fe
commit 06ea1f4c6d
2 changed files with 7 additions and 0 deletions
@@ -39,6 +39,7 @@ v5 已建:**77 例**(v4 20 例原样继承 + 57 例新增公开 Rodden-AA)
- `scripts/research/holdout_v5_baseline.py`:把 `minute_resolution_sweep` / `cluster_width_probe` / `futile_collect_stop_replay`(经 `guided_collect_holdout_replay.load_cases`)的模块常量指到指定数据集与独立输出文件,只跑 `baseline` 变体;不改这三个脚本。
- **硬红线 5**:`--dataset v4` 复跑,三个脚本共 7+12+6 组数字与 `minute_resolution_2026_09_14.json`、`cluster_width_2026_09_14.json`、`futile_collect_stop_replay_2026_09_29.json` 逐格一致(表在 `holdout_v5_build_2026_09_29.md` §3;JSON `holdout_v5_baseline_v4_all_2026_09_29.json`)。
- v5 全集:`holdout_v5_baseline_v5_all_2026_09_29.json`,数字见构建说明 §4。
- 发现并处理:`cluster_width_probe.run` 会顺手重写 `minute_resolution_2026_09_14.md` 里的勘误表(数值相同、格式 `15.0`→`15`)。已 `git checkout` 还原该文件,并在 `holdout_v5_baseline.py` 里把 `append_previous_errata` 置空,保证复跑不碰已发布文档。
| 半径 | 引擎先验头名(v4) | 六题回放头名(v4) | 六题后真值在区间(v4) | 六题后宽度中位(v4) | D1 六格真值在区间 truth / opposite |
| --- | --- | --- | --- | --- | --- |
+6
View File
@@ -66,6 +66,11 @@ def baseline_only(module: Any) -> None:
module.all_variants = lambda: [module.Variant("baseline")]
def keep_published_reports_untouched() -> None:
"""cluster_width_probe.run rewrites the errata table inside the 09-14 sweep report; never do that here."""
cwp.append_previous_errata = lambda *args, **kwargs: None
def run_sweep(dataset: Path, out_dir: Path, tag: str, radii: tuple[int, ...]) -> dict[str, Any]:
mrs.HOLDOUT = dataset
mrs.REPORT_MD = out_dir / f"holdout_v5_baseline_sweep_{tag}.md"
@@ -80,6 +85,7 @@ def run_cluster_width(dataset: Path, out_dir: Path, tag: str, radii: tuple[int,
cwp.REPORT_MD = out_dir / f"holdout_v5_baseline_cluster_width_{tag}.md"
cwp.REPORT_JSON = out_dir / f"holdout_v5_baseline_cluster_width_{tag}.json"
baseline_only(cwp)
keep_published_reports_untouched()
payload = cwp.run(argparse.Namespace(limit=0, radii=[str(r) for r in radii]))
replay = {
str(radius): payload["by_radius"][str(radius)]["baseline"].get("replay")