docs(rectification): bind sealed holdout metrics to the frozen scorer identity
The published six keys stay on the 2026-07-21 freeze. Current-tree sidecar numbers remain diagnostic and unlabeled as release metrics. Remove the unused v2 PILOT_REPORT_PATH. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -1038,3 +1038,11 @@
|
||||
- v3 `source_audit_status` 仍为 `invalidated_after_replay`。未改已发布的 `minute_rectification_holdout_v3_report.json` 数字。
|
||||
- v4 intake `day_precision_events_per_case=3` 未放宽:20 例中仅 4 例入列(`barack_obama_1961_aa_v4_holdout`、`paul_ryan_1970_aa_v4_holdout`、`sean_lennon_1975_aa_v4_holdout`、`kurt_cobain_1967_aa_v4_holdout`)。其余 16 例在补齐 review safeguards 后仍为 `insufficient_day_precision_events`;Robbins/Takamoto 校正后仍不够 3 个 day 事件。intake 保持 `production_tuning_allowed=false`、`verified_minute_claim_allowed=false`。
|
||||
- 诊断回放文件:`minute_rectification_holdout_v3_post_audit_diagnostic.json` / `_report.json`。这是 post-audit diagnostic rerun,不是盲测,不可声称 release gate 通过,不可用于调参。官方 eval runner 因当前树 scoring 文件哈希已偏离 2026-07-21 v3 freeze 而 `implementation_hash_matches=false`、trial_count=0;本任务未改 `frozen_scoring.files`。确认门仍关闭。
|
||||
|
||||
## 2026-08-29 - D1: 合同指标绑定冻结 scorer 身份;D3: 删除失效 PILOT_REPORT_PATH
|
||||
|
||||
- 六个既有键未改:`status=not_ready`、`valid_public_aa_cases=20`、`required_cases=20`、`top_1_rate=0.15`、`confirmation_coverage_rate=0.0`。
|
||||
- 新增 `metrics_produced_by.implementation_sha256=f41c298…`(2026-07-21 冻结 scorer,来自 v3 report);`current_tree_scorer.implementation_sha256=99730c8…` 且 `matches_metrics_scorer=false`、官方 runner `trial_count=0`(来自 post-audit diagnostic report)。
|
||||
- sidecar `current_tree_unfrozen_diagnostic` 记录当前树非盲诊断 `top_1_rate=0.45` / `top_3_rate=0.50` / `mean_absolute_minute_error=5.8`,并标明非盲、结果已被看过、scorer 未冻结、source audit 未过,不得当作发布指标。
|
||||
- 删除 `scripts/rectification/sealed_holdout.py` 未引用的 `PILOT_REPORT_PATH`(仍指向已被 v3 取代的 v2 试点报告)。
|
||||
- 防复发:评测指标必须与产生它的 `implementation_sha256` 绑定记录。
|
||||
|
||||
Reference in New Issue
Block a user