research(rectification): holdout v5 — drop current-year events (harness cutoff 2026-09-14), index pages

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017eEAG8HD3mm8gsKXgk8uU8
This commit is contained in:
Jesse_Chen
2026-09-29 19:52:43 +08:00
co-authored by Claude Fable 5.1
parent 57118b2697
commit 8ceb94da02
6 changed files with 17 additions and 57 deletions
+1
View File
@@ -122,3 +122,4 @@ Do not open new product surfaces before at least one of these four lanes is clos
- 两个已被证伪的假设:加权重能拉开候选;簇合并把候选并没了(合并从未触发)。
- 未解决的只剩「区间内部并列」,目前唯一被证明有效的信息源是用户提供的带年月新经历。
- 2026-09-29 打分方法研究(脚手架,判定待 v5):`<repo>/docs/research/rectification_scoring_research_2026_09_29.md`——似然比校准权重 / 缺席证据 / 精度追问三条流水线已在 v4 上跑通,所有判定 `pending_v5`;任务书 `docs/tasks/TASK-rectification-scoring-research-20260929.md`,前置 `TASK-rectification-holdout-expansion-20260929.md`。
- 2026-09-29 起评价集换 **v5**(77 例公开 AA,`references/real_case_calibration/minute_rectification_holdout_v5.json`):协议 `holdout_v5_protocol_2026_09_29.md`、构建与基线成绩单 `holdout_v5_build_2026_09_29.md`。后续打分判定以 v5 为准,v4 数字只作历史对照。
@@ -95,6 +95,12 @@
- 新增答错容错:答错 1 题,真值在区间 98–100%;答错 2 题,±30 / ±60 上 7–10% 的回放会挤出真值(两道反答 = 8 分 = 淘汰线)。
- 全文见 `docs/research/rectification_offline_research_2026_09_26.md`。
## 8. v5 已建,后续判定以 v5 为准(2026-09-29)
- 本页所有「过门 / 不过门」都在 20 例上判,1 例 = 5 个百分点,判定在噪声线上(任务书 `TASK-rectification-holdout-expansion-20260929.md`)。
- 开放评价集已扩到 **v5:77 例**(v4 的 20 例原样继承 + 57 例新增公开 Rodden-AA),每例 ≥7 件带引文的事件、≥4 领域;协议见 `holdout_v5_protocol_2026_09_29.md`,构建说明与基线成绩单见 `holdout_v5_build_2026_09_29.md`。
- v5 中 v4 子集的基线数字与本页 §1 逐格一致(复现表在构建说明里)。**此后任何打分方案的判定必须在 v5 上做**;`TASK-rectification-scoring-research-20260929.md` 的 R-A / R-B / R-C 以 v5 为准。
## 9. 后续(2026-09-29 打分方法研究单,脚手架阶段)
- §1.3「加权重类改法全部不过门」说的是**人拍权重**。2026-09-29 研究单换了路线:每条规则的权重由数据估(似然比,leave-one-case-out),允许负权重(只奖不罚是结构性问题),KP / Pranapada / D60 作为特征进入由数据定权重;另测「缺席证据」与「对已说事件追问月份」。
+1 -1
View File
@@ -255,7 +255,7 @@
| `TASK-self-edit-avatar-menu-20260928.md` | `PROGRESS-self-edit-avatar-menu-20260928.md` | **本人可编辑 + 各页头像菜单 + 报告页去说明**:`/people` 本人可编辑出生资料(BUG-1081,`ab6c55f8` 漏掉);次级页头像弹同一账户菜单、弹窗项跳 `/?account=`;我的报告删顶部说明与统计 | 已实现待验收(分支 `codex/self-edit-avatar-menu-20260928`,未推送) | BUG-1081 |
| `TASK-mobile-chart-and-confirmed-edit-20260929.md` | `PROGRESS-mobile-chart-confirmed-edit-20260929.md` | **手机星盘 + 确认时间可改 + 天空定格**:iPhone 星盘被宽表撑出屏幕(BUG-1083);confirmed 状态改声明字段也重置排盘时间(推翻 BUG-264 约定);「那一刻的天空」改为重放汇聚→定格→右上角分享;去掉报告星图封面 | 已验收(Claude 2026-09-29),已推 staging | BUG-1083 |
| `TASK-rectification-futile-collect-stop-20260929.md` | `PROGRESS-rectification-futile-collect-stop-20260929.md` | **生时校正停掉无效补经历循环(止血)**:打字经历不收窄(BUG-560 后果),流程却一路索要,真机 22 件整窗不动;采集只为开闸、门开后只问点选卡问完即出卡;交付正文去吻合率;多段时旁白误报「范围没变」;交付轮带采集题 | 已验收(Claude 2026-09-29 合并验收:全量前端 4302/24 与基线同 24 条环境失败、tsc 0、lint 0 error、`/` Static、gzip +0.16%、快速门 pytest 1001 passed、两份回放复跑一致),待部署核对 | BUG-1084~1087 |
| `TASK-rectification-holdout-expansion-20260929.md` | — | **开放评价集扩到 ≥60 例(v5)**:所有打分判定都在 20 例上做,1 例 = 5pp,KP / 精度闸 / V1n 的 no_benefit 都只差 1–3 例。只用 Rodden AA 公开名人,事件人工核对出处,分层(年代 / 纬度 / 南半球 / 跨午夜 / UTC+8),v4 不动;基线成绩单 v4 子集须与已发布数字逐格一致。研究单的判定以 v5 为准 | 待领取 | — |
| `TASK-rectification-holdout-expansion-20260929.md` | `PROGRESS-rectification-holdout-expansion-20260929.md` | **开放评价集扩到 ≥60 例(v5)**:所有打分判定都在 20 例上做,1 例 = 5pp,KP / 精度闸 / V1n 的 no_benefit 都只差 1–3 例。只用 Rodden AA 公开名人,事件人工核对出处,分层(年代 / 纬度 / 南半球 / 跨午夜 / UTC+8),v4 不动;基线成绩单 v4 子集须与已发布数字逐格一致。研究单的判定以 v5 为准 | 已实现待验收(v5 77 例:v4 20 + 新增 57;UTC+8 配额 5/6 未达,见 PROGRESS) | 分支 `codex/rectification-holdout-expansion-20260929`(BUG-1090) |
| `TASK-rectification-scoring-research-20260929.md` | `PROGRESS-rectification-scoring-research-20260929.md` | **打分方法研究(离线)**:R-A 似然比校准权重(leave-one-case-out,KP / Pranapada / D60 作为特征由数据定权重、允许负权重)、R-B 缺席证据(口述时间线覆盖时段内的空白年扣分,必测漏说稳健性)、R-C 精度追问可行性(对已说事件追问月份,算在定向追问 2 条额度内)。判定必须在 v5 上做;有收益才立实现单并按 ERR-110 重冻结 | 脚手架已验收(Claude 09-29:对账 60 case-radius 0 差、JSON 复跑逐字节一致、冻结文件零改动、117 测试全绿;已合入 staging);R-A / R-B / R-C 判定待 v5 | 分支 `codex/rectification-scoring-research-20260929`(BUG-1091 investigating) |
| `TASK-rectification-typed-event-scoring-research-20260929.md` | `PROGRESS-rectification-typed-event-research-20260929.md` | **打字经历按选择题规则计分(离线研究,不上线)**:计分通道不对称 + 已入账年份挡题;R0 学业质量题措辞 / 年精度显示成 1 月(冻结文件,需重新冻结) | 已验收(Claude 2026-09-29 合并验收:全量前端 4302/24 与基线同 24 条环境失败、tsc 0、lint 0 error、`/` Static、gzip +0.16%、快速门 pytest 1001 passed、两份回放复跑一致),待部署核对 | BUG-1088、1089 |
| `TASK-report-reader-polish-20260929.md` | `PROGRESS-report-reader-polish-20260929.md` | **报告页打磨**:生成入口挪进页面主体(删标题栏按钮)、详情页到底部按钮(懒渲染一次到底)、导出按钮带文字、目录一级/二级分层、去掉「字段」与 RL/NL 缩写表头、状态列同义重复去重、报告表格淡底色、分块导出显示真实文件大小 | Claude 直接执行并自验(tsc 0、lint 0 error、全量 fail 与基线同 24 条、`/` Static、gzip +7 B、快速门 Python 1000 passed),已部署 staging `d7772011`,真机欠 | BUG-1092~1094 |
@@ -31,16 +31,16 @@
"summary": {
"cases": 77,
"inherited_from_v4": 20,
"events_total": 966,
"events_total": 964,
"events_by_precision": {
"day": 244,
"month": 143,
"year": 579
"year": 577
},
"events_by_domain": {
"career": 321,
"career": 320,
"education": 117,
"family": 131,
"family": 130,
"finance": 35,
"health_pressure": 97,
"relationship": 193,
@@ -12133,19 +12133,6 @@
"quote": "He joined the MLS club Inter Miami in 2023 and led them to their first MLS Cup victory in 2025, while also winning the league MVP award in 2024 and 2025.",
"independent_of_birth_source": true
}
},
{
"id": "lionel_messi_1987_family_2026",
"domain": "family",
"date": "2026",
"precision": "year",
"description": "Father Jorge died.",
"source": {
"publisher": "Wikipedia",
"url": "https://en.wikipedia.org/wiki/Lionel_Messi",
"quote": "His father Jorge was his agent and manager from 2000 until 2026, when he died at the age of 68.",
"independent_of_birth_source": true
}
}
],
"candidate_radii_minutes": [
@@ -17033,19 +17020,6 @@
"quote": "He resigned in 2018, but returned to the club in 2019, and won another La Liga title and a Supercopa de España, before leaving again in 2021.",
"independent_of_birth_source": true
}
},
{
"id": "zinedine_zidane_1972_career_2026",
"domain": "career",
"date": "2026",
"precision": "year",
"description": "Became head coach of the France national team.",
"source": {
"publisher": "Wikipedia",
"url": "https://en.wikipedia.org/wiki/Zinedine_Zidane",
"quote": "In 2026, he returned to coaching, succeeding Didier Deschamps as the new head coach of the France national team. == Early life and career == Zinedine Yazid Zidane was born on 23 June 1972 in La Castellane, a neighbourhood in the 16th arrondissement of Marseille, in Southern France.",
"independent_of_birth_source": true
}
}
],
"candidate_radii_minutes": [
@@ -7227,19 +7227,6 @@
"quote": "He joined the MLS club Inter Miami in 2023 and led them to their first MLS Cup victory in 2025, while also winning the league MVP award in 2024 and 2025.",
"independent_of_birth_source": true
}
},
{
"id": "lionel_messi_1987_family_2026",
"domain": "family",
"date": "2026",
"precision": "year",
"description": "Father Jorge died.",
"source": {
"publisher": "Wikipedia",
"url": "https://en.wikipedia.org/wiki/Lionel_Messi",
"quote": "His father Jorge was his agent and manager from 2000 until 2026, when he died at the age of 68.",
"independent_of_birth_source": true
}
}
]
},
@@ -10967,19 +10954,6 @@
"quote": "He resigned in 2018, but returned to the club in 2019, and won another La Liga title and a Supercopa de España, before leaving again in 2021.",
"independent_of_birth_source": true
}
},
{
"id": "zinedine_zidane_1972_career_2026",
"domain": "career",
"date": "2026",
"precision": "year",
"description": "Became head coach of the France national team.",
"source": {
"publisher": "Wikipedia",
"url": "https://en.wikipedia.org/wiki/Zinedine_Zidane",
"quote": "In 2026, he returned to coaching, succeeding Didier Deschamps as the new head coach of the France national team. == Early life and career == Zinedine Yazid Zidane was born on 23 June 1972 in La Castellane, a neighbourhood in the 16th arrondissement of Marseille, in Southern France.",
"independent_of_birth_source": true
}
}
]
}
+5
View File
@@ -33,6 +33,7 @@ MIN_DOMAINS = 4
MIN_CASES = 60
DOMAINS = {"education", "relocation", "relationship", "career", "finance", "health_pressure", "family", "appearance", "occupation"}
PRECISION_FORMAT = {"year": 4, "month": 7, "day": 10}
EVENT_CUTOFF = "2026-09-14" # TODAY of minute_resolution_sweep / cluster_width_probe
def load(path: Path) -> Any:
@@ -71,6 +72,10 @@ def validate_event(case_id: str, event: dict[str, Any], problems: list[str]) ->
date = str(event.get("date", ""))
if precision not in PRECISION_FORMAT or len(date) != PRECISION_FORMAT[precision]:
problems.append(f"{case_id}: date {date!r} does not match precision {precision!r}")
# The replay harnesses evaluate at a fixed date (2026-09-14); the scoring contract
# rejects events after "today", and a year-precision event resolves inside its year.
if date[:4] >= EVENT_CUTOFF[:4] and (precision == "year" or date > EVENT_CUTOFF):
problems.append(f"{case_id}: event {event.get('id')} dated {date} is not before the harness cutoff {EVENT_CUTOFF}")
source = event.get("source") or {}
if not source.get("url") or not source.get("independent_of_birth_source"):
problems.append(f"{case_id}: event {event.get('id')} lacks an independent source url")