docs(research): measure minute-resolution scoring; no variant clears all radii
Build holdout v4 from the public AA set, correct the two v3 dates, and sweep R1–R5 plus pairs offline. No production scoring defaults change. No implementation brief: delivered width stays the full window on every radius.
This commit is contained in:
jesse-ux
@@ -0,0 +1,49 @@
|
||||
# Holdout v4 构建说明（2026-09-14）
|
||||
|
||||
- 口径：ayanamsa `raman`，node mode `mean`。
|
||||
- 父集：`references/real_case_calibration/minute_rectification_holdout_v3.json`（20 例公开 Rodden-AA）。
|
||||
- 产物：`references/real_case_calibration/minute_rectification_holdout_v4.json`
|
||||
- 生成脚本：`scripts/research/holdout_v4_build.py`
|
||||
- 隐私：只用公开名人传记，不用真实用户资料。
|
||||
|
||||
## 为什么 v3 不能当尺度改动的否决门
|
||||
|
||||
| 项 | v3 | 真实会话 / 本产品 |
|
||||
| --- | --- | --- |
|
||||
| 每例事件 | 恰好 3 | 5–18 |
|
||||
| 每例领域 | 2–3 | ≥4 才像一次问完的会话 |
|
||||
| 搜索半径 | 全部 ±10 | 产品有 ±15/30/60/120 |
|
||||
| 来源审计 | `invalidated_after_replay`（两处日期错） | 应用工作日期必须是核实过的 |
|
||||
| 事件精度 | 多为 year | 会话里 year 也多，但 3 条 year 事件几乎没有分钟方差 |
|
||||
|
||||
BUG-560 用 v3 否决 offset（覆盖 12/20）时，窗口本身只有 21 分钟、每例 3 件事。底座 11.5 对变化项 2.1 的比例在这种样本上几乎看不到差异。v4 用来量尺度，不是新的封存盲测门。
|
||||
|
||||
## v3 日期修正（任务 0）
|
||||
|
||||
| 事件 | 提交值 | 核实值 | 来源 |
|
||||
| --- | --- | --- | --- |
|
||||
| `robbins_marriage_1968` | 1968（年） | 1967-03-10 | Wikipedia *John Robbins (author)* |
|
||||
| `takamoto_joined_disney_1947` | 1947（年） | 1945（年） | Wikipedia *Iwao Takamoto* |
|
||||
|
||||
v3 的 `source_audit_status` 改为 `corrected_known_date_errors`。它仍是每例 3 件事，**不是发布门**。
|
||||
|
||||
## v4 协议
|
||||
|
||||
1. 出生时间与地点原样继承 v3 的 Astro-Databank AA 记录。
|
||||
2. 每例补到 ≥7 件带年月事件、≥4 个领域。事件来自 Wikipedia / Britannica / 奥斯卡官方 / 白宫档案等公开页，且 `independent_of_birth_source=true`。
|
||||
3. 工作日期用核实过的公历，不再保留已被审计否定的盲测错年。
|
||||
4. 三档半径 ±10 / ±30 / ±60 写在数据集上，由测量脚本分别跑。真实分钟所在簇随打分方案变，记在 sweep 结果里，不写死在 holdout 里。
|
||||
5. `truth_hidden_from_ranker=false`：这是开放评价集，用来看尺度，不是第二份封存盲测。
|
||||
|
||||
## 与 v3 的差异
|
||||
|
||||
- 20 例名单不变。
|
||||
- 事件从 3 增到 ≥7，领域从 2–3 增到 ≥4。
|
||||
- 两处错年按核实值改正，并进入 v4 工作日期。
|
||||
- 明确三档半径和本仓口径。
|
||||
|
||||
## 基线成绩单
|
||||
|
||||
当前线上算法在 v4 上的五个指标见 `docs/research/minute_resolution_2026_09_14.md`。那份表才是后续改法的对照，不是 v3 的 12/20。
|
||||
|
||||
摘要：±10 / ±30 / ±60 头名簇命中 0.35 / 0.15 / 0.10，区间覆盖 1.00，宽度中位等于整窗（21 / 61 / 121），并列率 0.05 / 0.05 / 0.00。
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,181 @@
|
||||
# 分钟分辨率打分尺度测量（2026-09-14）
|
||||
|
||||
- 口径：ayanamsa `raman`，node mode `mean`。不得与上游 true-node 数字直接对比。
|
||||
- 数据：`references/real_case_calibration/minute_rectification_holdout_v4.json`，20 例公开 Rodden-AA，每例 ≥7 件事、≥4 个领域。
|
||||
- 半径：±10, ±30, ±60，步长 2 分钟。
|
||||
- 性质：离线测量。生产 `active_rectification_event_engine.py` / `scoring_service.py` / `candidate_contrast.py` 默认值未改。
|
||||
- D1：KP 宫头子主计分方向已拍板；本文件只报告权重收益，不改线上默认。
|
||||
- 过门标准（任务书）：命中率与区间覆盖率不降、并列率下降、区间宽度中位数下降。三档半径要一致才算「有收益」。
|
||||
|
||||
## 结论
|
||||
|
||||
**没有改法在 ±10 / ±30 / ±60 同时过门，不立实现单。** 线上默认权重保持不动。
|
||||
|
||||
读数字时先看这件事：三个半径上，所有方案的交付区间宽度中位数都等于整个搜索窗（21 / 61 / 121）。公开簇并集铺满了窗口，所以「去底座 / 加权 / 换签名」几乎改不了用户看到的范围卡宽度。这是本轮最硬的结果。
|
||||
|
||||
| 改法 | 判定 | 要点 |
|
||||
| --- | --- | --- |
|
||||
| **R1 分盘除数** | 不确定 | ±10 头名簇命中 0.35→0.45（`len` / `sqrt`），宽窗上消失。宽度不变。 |
|
||||
| **R2 去底座** | 无收益 | 三档半径命中和宽度都不动；引擎熵有下降（±10：2.56→2.39），但过门不看熵。 |
|
||||
| **R3 KP 子主计分** | 无收益 / 不确定 | 20/20 例在 ±10 窗内宫头子主都变了（5–9 次），不是「窗内不变」的假阴性。±10 命中从 0.35 降到 0.20（权重 1.0），权重越大越差。产品方向仍有效，**默认权重不能直接上线**。 |
|
||||
| **R4 年精度边际** | 无收益 / 不确定 | `lse` 与基线几乎相同。`max` 在 ±10 命中到 0.50，但并列率 0.05→0.50，把峰值摊成平手。 |
|
||||
| **R5 动态签名层** | 无收益 | 命中下降或持平，熵上升。 |
|
||||
| 两两组合 | 无收益或不确定 | 没有一组三档半径同时过门。`R1+R2` 只在 ±10 复制了 R1 的命中提升。 |
|
||||
|
||||
基线本身：真分钟从未被挤出交付区间（覆盖率 1.00）；引擎头名簇命中只有 0.35 / 0.15 / 0.10。六题真值方向回放后头名 0.80 / 0.55 / 0.35——会话里「问完仍并列」主要不是 holdout 上的引擎头名问题，而是**范围卡宽度等于整窗**。
|
||||
|
||||
若下一轮还做，优先查「为什么公开簇并集永远铺满搜索窗」（签名层在 ±10 内是否几乎每分钟一组，或 `cap_clusters_by_adjacent_merge` 把弱簇并回整窗），而不是继续加权重。R3 若要再试，需要单独标定，且不得降低 ±10 命中。
|
||||
|
||||
### 分方案判定明细
|
||||
|
||||
| 改法 | 判定 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| R1@len | **uncertain** | ±10:benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R1@sqrt | **uncertain** | ±10:benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R1@fixed2 | **uncertain** | ±10:benefit；±30:benefit；±60:no_benefit |
|
||||
| R2 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R3@0.5 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R3@1.0 | **uncertain** | ±10:no_benefit；±30:benefit；±60:no_benefit |
|
||||
| R3@2.0 | **uncertain** | ±10:no_benefit；±30:benefit；±60:no_benefit |
|
||||
| R4@max | **uncertain** | ±10:uncertain；±30:uncertain；±60:uncertain |
|
||||
| R4@lse | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R5 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R1+R2 | **uncertain** | ±10:benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R1+R3 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R1+R4 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R1+R5 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R2+R3 | **uncertain** | ±10:no_benefit；±30:benefit；±60:no_benefit |
|
||||
| R2+R4 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R2+R5 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| R3+R4 | **uncertain** | ±10:no_benefit；±30:benefit；±60:no_benefit |
|
||||
| R3+R5 | **uncertain** | ±10:no_benefit；±30:benefit；±60:no_benefit |
|
||||
| R4+R5 | **no_benefit** | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
|
||||
## 基线成绩单
|
||||
|
||||
| 半径 | N | 真分钟在头名簇 | 真分钟在交付区间 | 区间宽度中位 | 并列率 | 引擎熵 | 真值被挤出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| ±10 | 20 | 0.35 | 1 | 21 | 0.05 | 2.559 | 0 |
|
||||
| ±30 | 20 | 0.15 | 1 | 61 | 0.05 | 4.038 | 0 |
|
||||
| ±60 | 20 | 0.1 | 1 | 121 | 0 | 4.992 | 0 |
|
||||
|
||||
## 分半径指标
|
||||
|
||||
### ±10 分钟
|
||||
|
||||
| 方案 | 头名簇命中 | 区间覆盖 | 宽度中位 | 并列率 | 熵0 | 熵6 | 挤出 | 过窄 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| baseline | 0.35 | 1 | 21 | 0.05 | 2.559 | 2.141 | 0 | 0 |
|
||||
| R1@len | 0.45 | 1 | 21 | 0.05 | 2.554 | 2.139 | 0 | 0 |
|
||||
| R1@sqrt | 0.45 | 1 | 21 | 0.05 | 2.554 | 2.139 | 0 | 0 |
|
||||
| R1@fixed2 | 0.4 | 1 | 21 | 0.05 | 2.559 | 2.141 | 0 | 0 |
|
||||
| R2 | 0.35 | 1 | 21 | 0.05 | 2.387 | 2.034 | 0 | 0 |
|
||||
| R3@0.5 | 0.25 | 1 | 21 | 0 | 2.56 | 2.142 | 0 | 0 |
|
||||
| R3@1.0 | 0.2 | 1 | 21 | 0 | 2.56 | 2.143 | 0 | 0 |
|
||||
| R3@2.0 | 0.15 | 1 | 21 | 0 | 2.557 | 2.142 | 0 | 0 |
|
||||
| R4@max | 0.5 | 1 | 21 | 0.5 | 2.56 | 2.143 | 0 | 0 |
|
||||
| R4@lse | 0.35 | 1 | 21 | 0.05 | 2.559 | 2.141 | 0 | 0 |
|
||||
| R5 | 0.25 | 1 | 21 | 0.05 | 2.648 | 2.194 | 0 | 0 |
|
||||
| R1+R2 | 0.45 | 1 | 21 | 0.05 | 2.356 | 2.025 | 0 | 0 |
|
||||
| R1+R3 | 0.25 | 1 | 21 | 0 | 2.556 | 2.142 | 0 | 0 |
|
||||
| R1+R4 | 0.35 | 1 | 21 | 0.05 | 2.554 | 2.139 | 0 | 0 |
|
||||
| R1+R5 | 0.35 | 1 | 21 | 0.05 | 2.643 | 2.192 | 0 | 0 |
|
||||
| R2+R3 | 0.2 | 1 | 21 | 0 | 2.485 | 2.088 | 0 | 0 |
|
||||
| R2+R4 | 0.35 | 1 | 21 | 0.05 | 2.386 | 2.035 | 0 | 0 |
|
||||
| R2+R5 | 0.25 | 1 | 21 | 0.05 | 2.47 | 2.104 | 0 | 0 |
|
||||
| R3+R4 | 0.2 | 1 | 21 | 0 | 2.56 | 2.143 | 0 | 0 |
|
||||
| R3+R5 | 0.15 | 1 | 21 | 0 | 2.649 | 2.196 | 0 | 0 |
|
||||
| R4+R5 | 0.25 | 1 | 21 | 0.05 | 2.648 | 2.195 | 0 | 0 |
|
||||
|
||||
### ±30 分钟
|
||||
|
||||
| 方案 | 头名簇命中 | 区间覆盖 | 宽度中位 | 并列率 | 熵0 | 熵6 | 挤出 | 过窄 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| baseline | 0.15 | 1 | 61 | 0.05 | 4.038 | 2.576 | 0 | 0 |
|
||||
| R1@len | 0.15 | 1 | 61 | 0.05 | 4.03 | 2.576 | 0 | 0 |
|
||||
| R1@sqrt | 0.15 | 1 | 61 | 0.05 | 4.03 | 2.576 | 0 | 0 |
|
||||
| R1@fixed2 | 0.2 | 1 | 61 | 0.05 | 4.037 | 2.576 | 0 | 0 |
|
||||
| R2 | 0.15 | 1 | 61 | 0.05 | 3.915 | 2.546 | 0 | 0 |
|
||||
| R3@0.5 | 0.1 | 1 | 61 | 0 | 4.039 | 2.577 | 0 | 0 |
|
||||
| R3@1.0 | 0.15 | 1 | 61 | 0 | 4.039 | 2.577 | 0 | 0 |
|
||||
| R3@2.0 | 0.15 | 1 | 61 | 0 | 4.036 | 2.577 | 0 | 0 |
|
||||
| R4@max | 0.2 | 1 | 61 | 0.25 | 4.039 | 2.578 | 0 | 0 |
|
||||
| R4@lse | 0.15 | 1 | 61 | 0.05 | 4.038 | 2.576 | 0 | 0 |
|
||||
| R5 | 0.1 | 1 | 61 | 0.05 | 4.096 | 2.576 | 0 | 0 |
|
||||
| R1+R2 | 0.15 | 1 | 61 | 0.05 | 3.874 | 2.541 | 0 | 0 |
|
||||
| R1+R3 | 0.1 | 1 | 61 | 0 | 4.033 | 2.577 | 0 | 0 |
|
||||
| R1+R4 | 0.15 | 1 | 61 | 0.05 | 4.03 | 2.576 | 0 | 0 |
|
||||
| R1+R5 | 0.1 | 1 | 61 | 0.05 | 4.088 | 2.576 | 0 | 0 |
|
||||
| R2+R3 | 0.15 | 1 | 61 | 0 | 3.973 | 2.557 | 0 | 0 |
|
||||
| R2+R4 | 0.15 | 1 | 61 | 0.05 | 3.91 | 2.547 | 0 | 0 |
|
||||
| R2+R5 | 0.1 | 1 | 61 | 0.05 | 3.97 | 2.546 | 0 | 0 |
|
||||
| R3+R4 | 0.15 | 1 | 61 | 0 | 4.039 | 2.578 | 0 | 0 |
|
||||
| R3+R5 | 0.15 | 1 | 61 | 0 | 4.097 | 2.577 | 0 | 0 |
|
||||
| R4+R5 | 0.1 | 1 | 61 | 0.05 | 4.096 | 2.576 | 0 | 0 |
|
||||
|
||||
### ±60 分钟
|
||||
|
||||
| 方案 | 头名簇命中 | 区间覆盖 | 宽度中位 | 并列率 | 熵0 | 熵6 | 挤出 | 过窄 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| baseline | 0.1 | 1 | 121 | 0 | 4.992 | 2.678 | 0 | 0 |
|
||||
| R1@len | 0.1 | 1 | 121 | 0 | 4.986 | 2.677 | 0 | 0 |
|
||||
| R1@sqrt | 0.1 | 1 | 121 | 0 | 4.987 | 2.677 | 0 | 0 |
|
||||
| R1@fixed2 | 0.1 | 1 | 121 | 0 | 4.992 | 2.678 | 0 | 0 |
|
||||
| R2 | 0.1 | 1 | 121 | 0 | 4.918 | 2.668 | 0 | 0 |
|
||||
| R3@0.5 | 0.05 | 1 | 121 | 0 | 4.995 | 2.678 | 0 | 0 |
|
||||
| R3@1.0 | 0.05 | 1 | 121 | 0 | 4.996 | 2.678 | 0 | 0 |
|
||||
| R3@2.0 | 0.1 | 1 | 121 | 0 | 4.995 | 2.678 | 0 | 0 |
|
||||
| R4@max | 0.1 | 1 | 121 | 0.2 | 4.993 | 2.678 | 0 | 0 |
|
||||
| R4@lse | 0.1 | 1 | 121 | 0 | 4.992 | 2.678 | 0 | 0 |
|
||||
| R5 | 0.1 | 1 | 121 | 0 | 5.033 | 2.677 | 0 | 0 |
|
||||
| R1+R2 | 0.1 | 1 | 121 | 0 | 4.903 | 2.667 | 0 | 0 |
|
||||
| R1+R3 | 0.05 | 1 | 121 | 0 | 4.992 | 2.678 | 0 | 0 |
|
||||
| R1+R4 | 0.1 | 1 | 121 | 0 | 4.987 | 2.677 | 0 | 0 |
|
||||
| R1+R5 | 0.1 | 1 | 121 | 0 | 5.027 | 2.677 | 0 | 0 |
|
||||
| R2+R3 | 0.05 | 1 | 121 | 0 | 4.954 | 2.671 | 0 | 0 |
|
||||
| R2+R4 | 0.1 | 1 | 121 | 0 | 4.919 | 2.669 | 0 | 0 |
|
||||
| R2+R5 | 0.1 | 1 | 121 | 0 | 4.956 | 2.668 | 0 | 0 |
|
||||
| R3+R4 | 0.1 | 1 | 121 | 0 | 4.996 | 2.679 | 0 | 0 |
|
||||
| R3+R5 | 0.05 | 1 | 121 | 0 | 5.037 | 2.678 | 0 | 0 |
|
||||
| R4+R5 | 0.1 | 1 | 121 | 0 | 5.033 | 2.677 | 0 | 0 |
|
||||
|
||||
## R3 KP 窗内变化
|
||||
|
||||
只统计 ±10 窗内目标宫宫头子主至少变化一次的例子。
|
||||
|
||||
| 例子 | 窗内变化次数 | 计入 R3 |
|
||||
| --- | ---: | ---: |
|
||||
| barack_obama_1961_aa_v4_holdout | 8 | yes |
|
||||
| angelina_jolie_1975_aa_v4_holdout | 8 | yes |
|
||||
| tiger_woods_1975_aa_v4_holdout | 7 | yes |
|
||||
| elizabeth_taylor_1932_aa_v4_holdout | 9 | yes |
|
||||
| elizabeth_montgomery_1933_aa_v4_holdout | 7 | yes |
|
||||
| pablo_picasso_1881_aa_v4_holdout | 8 | yes |
|
||||
| sigmund_freud_1856_aa_v4_holdout | 5 | yes |
|
||||
| salma_hayek_1966_aa_v4_holdout | 7 | yes |
|
||||
| steve_reich_1936_aa_v4_holdout | 7 | yes |
|
||||
| albert_brooks_1947_aa_v4_holdout | 7 | yes |
|
||||
| paul_ryan_1970_aa_v4_holdout | 8 | yes |
|
||||
| tom_kennedy_1927_aa_v4_holdout | 7 | yes |
|
||||
| andrew_windsor_1960_aa_v4_holdout | 8 | yes |
|
||||
| sean_lennon_1975_aa_v4_holdout | 6 | yes |
|
||||
| joseph_kennedy_iii_1980_aa_v4_holdout | 9 | yes |
|
||||
| kurt_cobain_1967_aa_v4_holdout | 7 | yes |
|
||||
| john_robbins_1947_aa_v4_holdout | 8 | yes |
|
||||
| chad_everett_1937_aa_v4_holdout | 8 | yes |
|
||||
| bernd_eichinger_1949_aa_v4_holdout | 8 | yes |
|
||||
| iwao_takamoto_1925_aa_v4_holdout | 7 | yes |
|
||||
|
||||
## 方法
|
||||
|
||||
1. 每个例子先算生产静态盘（含 KP 观察），再按事件抽样日打生产分。
|
||||
2. R1 按 `*_domain_varga` 规则把 `2*len` 换成 `len` / `sqrt(len)` / 固定 2。
|
||||
3. R2 在事件级减去当前窗口最小值。
|
||||
4. R3 把目标宫 KP 子主/次子主与当时 Vim MD/AD/PD 比对，权重 0.5 / 1.0 / 2.0。
|
||||
5. R4 对年精度 12 个月样本取 max 或 log-sum-exp（T=1）。
|
||||
6. R5 按本例有证据的领域动态取签名层，去掉同日恒定的 `md`。
|
||||
7. 交付区间 = 公开簇成员分钟的并集。并列 = 头名簇分数并列。
|
||||
8. 熵曲线用生产探针 + 真值方向最优答，最多 6 题。
|
||||
|
||||
配对组合用 R1=`len`、R3=1.0、R4=`lse`。
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# 进度 · 分钟分辨率打分尺度研究（2026-09-14）
|
||||
|
||||
## 范围
|
||||
|
||||
- 分支：`codex/rectification-minute-resolution-research-20260914`（基于 `origin/staging` @ `9943a05a`）
|
||||
- 任务单：`docs/tasks/TASK-rectification-minute-resolution-research-20260914.md`
|
||||
- 结果：`docs/research/minute_resolution_2026_09_14.md` + 同名 `.json`
|
||||
- v4：`references/real_case_calibration/minute_rectification_holdout_v4.json`、`docs/research/holdout_v4_build_2026_09_14.md`
|
||||
- 脚本：`scripts/research/minute_resolution_sweep.py`（离线，不进 API）
|
||||
- 未改 `active_rectification_event_engine.py`、`scoring_service.py`、`candidate_contrast.py`、确认门、并列判据
|
||||
- 本单不占 BUG 号
|
||||
|
||||
## 完成
|
||||
|
||||
- 开工读了对照报告、BUG-325 / 560 / 570。D1 推翻的是 BUG-325「不得把 KP 观察计分」这一条；确认门仍不动。
|
||||
- 任务 0：v3 两处错年写入工作日期（Robbins 婚期 1967-03-10，Takamoto 入职迪士尼 1945），`source_audit_status=corrected_known_date_errors`。v4 仍是 20 例公开 AA，每例 ≥7 件事、≥4 个领域，半径 ±10/±30/±60。
|
||||
- 任务 1：五个改法单独测 + 两两组合。口径 ayanamsa=`raman`、node mode=`mean`。20/20 例跑完，错误 0。
|
||||
- R3：±10 窗内 KP 宫头子主 20/20 都变（5–9 次），没有「窗内不变」的例子被稀释。
|
||||
|
||||
## 验收数字
|
||||
|
||||
- 基线（v4，当前线上算法）：
|
||||
|
||||
| 半径 | 头名簇命中 | 区间覆盖 | 宽度中位 | 并列率 | 引擎熵 | 六题回放头名 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| ±10 | 0.35 | 1.00 | 21 | 0.05 | 2.56 | 0.80 |
|
||||
| ±30 | 0.15 | 1.00 | 61 | 0.05 | 4.04 | 0.55 |
|
||||
| ±60 | 0.10 | 1.00 | 121 | 0.00 | 4.99 | 0.35 |
|
||||
|
||||
- 所有方案的交付宽度中位数都等于整个搜索窗。真值挤出 = 0，过窄 = 0。
|
||||
- R1 `len`/`sqrt`：±10 命中 0.35→0.45，宽窗上无提升。
|
||||
- R2：命中/宽度不动，±10 熵 2.56→2.39。
|
||||
- R3 权重 1.0：±10 命中 0.35→0.20。
|
||||
- R4 `max`：±10 命中 0.50，并列率 0.05→0.50。
|
||||
- 判定：**没有改法三档半径同时过门，不立实现单。**
|
||||
- `python -m pytest tests/test_minute_resolution_research.py`：9 passed（本机 pytest 9.1；仓库 `minversion` 8.0）。
|
||||
- 本机无项目 `.venv`，未跑 `run_quality_gate.py --profile quick`。脚本与测试会进门禁路径。
|
||||
|
||||
## 未做
|
||||
|
||||
- 未改线上计分
|
||||
- 未把研究脚本接到 API
|
||||
- 未立实现单（过门失败）
|
||||
- 未做 PyJHora / VedAstro / jyotishganit 同一盘对照
|
||||
@@ -184,7 +184,7 @@
|
||||
|
||||
| `TASK-rectification-targeted-card-dead-20260913.md` | `PROGRESS-rectification-targeted-card-dead-20260913.md` | **P0**：定向补事卡在快照投影里拿不到 `choice_card`（承接焦点分支不重建 `choice_frame`），卡片看得见点不动、流程停在采集等待态；模型还会把定向题改写成口述题（BUG-669～671）。先于 tie-break 修复单执行 | 待验收 | `codex/rectification-targeted-card-dead-20260913` |
|
||||
|
||||
| `TASK-rectification-minute-resolution-research-20260914.md` | `PROGRESS-rectification-minute-resolution-research-20260914.md` | **研究单**：候选分不开的根因是打分尺度——窗口内恒定项 11.5 分 vs 随分钟变化项 2.125 分（≈5:1）。先修封存基准（v3 每例仅 3 件事且被标 invalidated）出 v4，再离线量五个改法：分盘除数、去底座、**KP 宫头子主计分（产品 09-14 拍板，推翻 BUG-325 一条红线）**、年精度事件改边际似然、聚类签名层对齐。有收益才立实现单 | 待执行 | `codex/rectification-minute-resolution-research-20260914` |
|
||||
| `TASK-rectification-minute-resolution-research-20260914.md` | `PROGRESS-rectification-minute-resolution-research-20260914.md` | **研究单**：候选分不开的根因是打分尺度——窗口内恒定项 11.5 分 vs 随分钟变化项 2.125 分（≈5:1）。先修封存基准（v3 每例仅 3 件事且被标 invalidated）出 v4，再离线量五个改法：分盘除数、去底座、**KP 宫头子主计分（产品 09-14 拍板，推翻 BUG-325 一条红线）**、年精度事件改边际似然、聚类签名层对齐。有收益才立实现单 | 待验收 | `codex/rectification-minute-resolution-research-20260914` |
|
||||
|
||||
| `TASK-rectification-tiebreak-before-card-20260914.md` | `PROGRESS-rectification-tiebreak-before-card-20260914.md` | **P0**：点「再答两道参考题」服务端 ok 但界面无反应（`requestTieBreak` 的 `loadCaseSnapshot()` 不带合并回调，新 turn 不入对话区）（BUG-685）；产品拍板放宽版 A——出交付卡前只要参考题可用且没用过就先问两道（不看分差），卡上收起按钮（BUG-686）；交付文案列出用户已拒答的线（BUG-687） | 待验收 | `codex/rectification-tiebreak-before-card-20260914` |
|
||||
|
||||
|
||||
@@ -4,13 +4,16 @@
|
||||
"description": "Sealed public Rodden-AA minute holdout for the event-specific fact ranker v4.",
|
||||
"truth_hidden_from_ranker": true,
|
||||
"frozen_before_replay": true,
|
||||
"source_audit_status": "invalidated_after_replay",
|
||||
"source_audit_status": "corrected_known_date_errors",
|
||||
"source_audit_note": "2026-09-14: the two known date errors were written into the working date fields. This file is still 3 events per case and is not a release gate. Use minute_rectification_holdout_v4.json for scale measurement.",
|
||||
"source_audit_issues": [
|
||||
{
|
||||
"event_id": "robbins_marriage_1968",
|
||||
"status": "corrected",
|
||||
"issue": "The cited public biography gives 1967-03-10, not 1968.",
|
||||
"submitted_date": "1968",
|
||||
"verified_date": "1967-03-10",
|
||||
"working_date": "1967-03-10",
|
||||
"cited_url": "https://www.johnrobbins.info/about/",
|
||||
"cited_url_outcome": "replaced",
|
||||
"cited_url_note": "The original about page has no marriage date.",
|
||||
@@ -18,9 +21,11 @@
|
||||
},
|
||||
{
|
||||
"event_id": "takamoto_joined_disney_1947",
|
||||
"status": "corrected",
|
||||
"issue": "Multiple public biographies give 1945, not 1947.",
|
||||
"submitted_date": "1947",
|
||||
"verified_date": "1945",
|
||||
"working_date": "1945",
|
||||
"cited_url": "https://en.wikipedia.org/wiki/Iwao_Takamoto",
|
||||
"cited_url_outcome": "kept",
|
||||
"cited_url_note": "The cited Wikipedia page states hired by Walt Disney Productions in 1945, not 1947."
|
||||
@@ -317,7 +322,7 @@
|
||||
"source": {"publisher": "Astro-Databank", "url": "https://www.astro.com/astro-databank/Robbins%2C_John", "rodden_rating": "AA", "record_type": "birth_record"}
|
||||
},
|
||||
"events": [
|
||||
{"id": "robbins_marriage_1968", "domain": "relationship", "date": "1968", "precision": "year", "submitted_date": "1968", "verified_date": "1967-03-10", "source_audit_note": "Blind working date remains submitted 1968. Verified 1967-03-10 from Wikipedia; original johnrobbins.info/about/ has no marriage date.", "description": "Married Deo Robbins.", "source": {"publisher": "Wikipedia", "url": "https://en.wikipedia.org/wiki/John_Robbins_(author)", "independent_of_birth_source": true}},
|
||||
{"id": "robbins_marriage_1968", "domain": "relationship", "date": "1967-03-10", "precision": "day", "submitted_date": "1968", "verified_date": "1967-03-10", "source_audit_note": "Working date corrected 2026-09-14 to the verified 1967-03-10 Wikipedia biography date.", "description": "Married Deo Robbins.", "source": {"publisher": "Wikipedia", "url": "https://en.wikipedia.org/wiki/John_Robbins_(author)", "independent_of_birth_source": true}},
|
||||
{"id": "robbins_diet_book_1987", "domain": "career", "date": "1987", "precision": "year", "description": "Published Diet for a New America.", "source": {"publisher": "Wikipedia", "url": "https://en.wikipedia.org/wiki/John_Robbins_(author)", "independent_of_birth_source": true}},
|
||||
{"id": "robbins_health_book_1996", "domain": "career", "date": "1996", "precision": "year", "description": "Published Reclaiming Our Health.", "source": {"publisher": "John Robbins", "url": "https://www.johnrobbins.info/books/", "independent_of_birth_source": true}}
|
||||
]
|
||||
@@ -362,7 +367,7 @@
|
||||
"source": {"publisher": "Astro-Databank", "url": "https://www.astro.com/astro-databank/Takamoto%2C_Iwao", "rodden_rating": "AA", "record_type": "birth_record"}
|
||||
},
|
||||
"events": [
|
||||
{"id": "takamoto_joined_disney_1947", "domain": "career", "date": "1947", "precision": "year", "submitted_date": "1947", "verified_date": "1945", "source_audit_note": "Blind working date remains submitted 1947. Cited Wikipedia states hired by Walt Disney Productions in 1945; URL kept.", "description": "Joined Walt Disney Animation Studios.", "source": {"publisher": "Wikipedia", "url": "https://en.wikipedia.org/wiki/Iwao_Takamoto", "independent_of_birth_source": true}},
|
||||
{"id": "takamoto_joined_disney_1947", "domain": "career", "date": "1945", "precision": "year", "submitted_date": "1947", "verified_date": "1945", "source_audit_note": "Working date corrected 2026-09-14 to the verified 1945 Wikipedia hiring year.", "description": "Joined Walt Disney Animation Studios.", "source": {"publisher": "Wikipedia", "url": "https://en.wikipedia.org/wiki/Iwao_Takamoto", "independent_of_birth_source": true}},
|
||||
{"id": "takamoto_hanna_barbera_1961", "domain": "career", "date": "1961", "precision": "year", "description": "Moved to Hanna-Barbera and began designing its characters.", "source": {"publisher": "Animation Magazine", "url": "https://www.animationmagazine.net/2007/01/animation-legend-iwao-takamoto-dies/", "independent_of_birth_source": true}},
|
||||
{"id": "takamoto_heart_attack_2007", "domain": "health_pressure", "date": "2007-01-08", "precision": "day", "description": "Died following a heart attack.", "source": {"publisher": "The New York Times", "url": "https://www.nytimes.com/2007/01/10/arts/10takamoto.html", "independent_of_birth_source": true}}
|
||||
]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,305 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build minute_rectification_holdout_v4 from the corrected v3 public AA set.
|
||||
|
||||
Adds dated public events so each case has at least 7 events and 4 domains.
|
||||
Does not use private user data. Working dates for the two v3 audit errors
|
||||
are the verified public dates.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
V3_PATH = ROOT / "references" / "real_case_calibration" / "minute_rectification_holdout_v3.json"
|
||||
V4_PATH = ROOT / "references" / "real_case_calibration" / "minute_rectification_holdout_v4.json"
|
||||
|
||||
RADII = (10, 30, 60)
|
||||
MIN_EVENTS = 7
|
||||
MIN_DOMAINS = 4
|
||||
|
||||
|
||||
def event(
|
||||
event_id: str,
|
||||
domain: str,
|
||||
date: str,
|
||||
precision: str,
|
||||
description: str,
|
||||
publisher: str,
|
||||
url: str,
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
"id": event_id,
|
||||
"domain": domain,
|
||||
"date": date,
|
||||
"precision": precision,
|
||||
"description": description,
|
||||
"source": {
|
||||
"publisher": publisher,
|
||||
"url": url,
|
||||
"independent_of_birth_source": True,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
WIKI = "Wikipedia"
|
||||
BRIT = "Encyclopaedia Britannica"
|
||||
OSCARS = "Academy of Motion Picture Arts and Sciences"
|
||||
|
||||
# Full event lists replace the 3-event v3 rows. Birth records stay from v3.
|
||||
EVENTS: dict[str, list[dict[str, Any]]] = {
|
||||
"barack_obama_1961_aa_v4_holdout": [
|
||||
event("obama_columbia_1983", "education", "1983", "year", "Graduated from Columbia University.", WIKI, "https://en.wikipedia.org/wiki/Barack_Obama"),
|
||||
event("obama_chicago_1985", "relocation", "1985", "year", "Moved to Chicago as a community organizer.", BRIT, "https://www.britannica.com/biography/Barack-Obama"),
|
||||
event("obama_harvard_law_1991", "education", "1991", "year", "Graduated from Harvard Law School.", WIKI, "https://en.wikipedia.org/wiki/Barack_Obama"),
|
||||
event("obama_wedding_1992_v4", "relationship", "1992-10-03", "day", "Married Michelle Robinson.", "Obama Foundation", "https://obama.org/about/administration/chicago-where-it-began/"),
|
||||
event("obama_illinois_senate_1996_v4", "career", "1996-11-05", "day", "Elected to the Illinois State Senate.", WIKI, "https://en.wikipedia.org/wiki/Illinois_Senate_career_of_Barack_Obama"),
|
||||
event("obama_malia_1998", "family", "1998-07-04", "day", "Daughter Malia was born.", WIKI, "https://en.wikipedia.org/wiki/Malia_Obama"),
|
||||
event("obama_us_senate_2004", "career", "2004-11-02", "day", "Elected to the United States Senate.", WIKI, "https://en.wikipedia.org/wiki/2004_United_States_Senate_election_in_Illinois"),
|
||||
event("obama_presidential_election_2008_v4", "career", "2008-11-04", "day", "Elected President of the United States.", "White House Archives", "https://obamawhitehouse.archives.gov/1600/presidents/barackobama/"),
|
||||
],
|
||||
"angelina_jolie_1975_aa_v4_holdout": [
|
||||
event("jolie_gia_1998", "career", "1998", "year", "Starred in the television film Gia.", WIKI, "https://en.wikipedia.org/wiki/Angelina_Jolie"),
|
||||
event("jolie_oscar_2000", "career", "2000-03-26", "day", "Won the Academy Award for supporting actress.", OSCARS, "https://www.oscars.org/oscars/ceremonies/2000"),
|
||||
event("jolie_thornton_2000", "relationship", "2000-05-05", "day", "Married Billy Bob Thornton.", BRIT, "https://www.britannica.com/biography/Angelina-Jolie"),
|
||||
event("jolie_maddox_2002", "family", "2002-03-10", "day", "Adopted Maddox from Cambodia.", WIKI, "https://en.wikipedia.org/wiki/Angelina_Jolie"),
|
||||
event("jolie_shiloh_2006", "family", "2006-05-27", "day", "Daughter Shiloh was born.", WIKI, "https://en.wikipedia.org/wiki/Shiloh_Jolie"),
|
||||
event("jolie_mastectomy_2013", "health_pressure", "2013", "year", "Underwent preventive double mastectomy treatment.", "The New York Times", "https://www.nytimes.com/2013/05/14/opinion/my-medical-choice.html"),
|
||||
event("jolie_marriage_2014", "relationship", "2014-08-23", "day", "Married Brad Pitt.", BRIT, "https://www.britannica.com/biography/Angelina-Jolie"),
|
||||
event("jolie_divorce_filed_2016", "relationship", "2016-09-19", "day", "Filed for divorce from Brad Pitt.", WIKI, "https://en.wikipedia.org/wiki/Angelina_Jolie"),
|
||||
],
|
||||
"tiger_woods_1975_aa_v4_holdout": [
|
||||
event("woods_stanford_1994", "education", "1994", "year", "Enrolled at Stanford University.", WIKI, "https://en.wikipedia.org/wiki/Tiger_Woods"),
|
||||
event("woods_turned_professional_1996", "career", "1996", "year", "Turned professional as a golfer.", "PGA Tour", "https://www.pgatour.com/player/08793/tiger-woods/bio"),
|
||||
event("woods_masters_1997", "career", "1997-04-13", "day", "Won the Masters Tournament.", WIKI, "https://en.wikipedia.org/wiki/1997_Masters_Tournament"),
|
||||
event("woods_marriage_2004", "relationship", "2004-10-05", "day", "Married Elin Nordegren.", BRIT, "https://www.britannica.com/biography/Tiger-Woods"),
|
||||
event("woods_sam_2007", "family", "2007-06-18", "day", "Daughter Sam was born.", WIKI, "https://en.wikipedia.org/wiki/Tiger_Woods"),
|
||||
event("woods_divorce_2010", "relationship", "2010-08-23", "day", "Divorce from Elin Nordegren was finalized.", WIKI, "https://en.wikipedia.org/wiki/Tiger_Woods"),
|
||||
event("woods_car_crash_2021", "health_pressure", "2021-02-23", "day", "Sustained serious injuries in a motor-vehicle crash.", WIKI, "https://en.wikipedia.org/wiki/Tiger_Woods"),
|
||||
],
|
||||
"elizabeth_taylor_1932_aa_v4_holdout": [
|
||||
event("taylor_moved_us_1939", "relocation", "1939", "year", "The family moved from London to the United States.", BRIT, "https://www.britannica.com/biography/Elizabeth-Taylor-American-actress"),
|
||||
event("taylor_national_velvet_1944", "career", "1944", "year", "Starred in National Velvet.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Taylor"),
|
||||
event("taylor_first_marriage_1950", "relationship", "1950-05-06", "day", "Married Conrad Hilton Jr.", BRIT, "https://www.britannica.com/biography/Elizabeth-Taylor-American-actress"),
|
||||
event("taylor_first_child_1953", "family", "1953-01-06", "day", "Her first child was born.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Taylor"),
|
||||
event("taylor_oscar_1961", "career", "1961-04-17", "day", "Won the Academy Award for Butterfield 8.", OSCARS, "https://www.oscars.org/oscars/ceremonies/1961"),
|
||||
event("taylor_burton_marriage_1964", "relationship", "1964-03-15", "day", "Married Richard Burton.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Taylor"),
|
||||
event("taylor_brain_surgery_1997", "health_pressure", "1997", "year", "Underwent surgery for a benign brain tumor.", BRIT, "https://www.britannica.com/biography/Elizabeth-Taylor-American-actress/Final-films-and-HIV-AIDS-activism"),
|
||||
],
|
||||
"elizabeth_montgomery_1933_aa_v4_holdout": [
|
||||
event("montgomery_academy_1951", "education", "1951", "year", "Studied at the American Academy of Dramatic Arts.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Montgomery"),
|
||||
event("montgomery_first_marriage_1954", "relationship", "1954", "year", "Married Frederick Gallatin Cammann.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Montgomery"),
|
||||
event("montgomery_marriage_asher_1963", "relationship", "1963", "year", "Married producer William Asher.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Montgomery"),
|
||||
event("montgomery_bewitched_1964", "career", "1964-09-17", "day", "Bewitched premiered on television.", WIKI, "https://en.wikipedia.org/wiki/Bewitched"),
|
||||
event("montgomery_william_jr_1964", "family", "1964-07-24", "day", "Son William Asher Jr. was born.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Montgomery"),
|
||||
event("montgomery_moved_into_series_fame_1964b", "career", "1966", "year", "Won a Golden Globe for Bewitched.", WIKI, "https://en.wikipedia.org/wiki/Elizabeth_Montgomery"),
|
||||
event("montgomery_cancer_1995", "health_pressure", "1995", "year", "Was diagnosed with colorectal cancer and died later that year.", "The New York Times", "https://www.nytimes.com/1995/05/19/obituaries/elizabeth-montgomery-62-star-of-the-tv-comedy-bewitched.html"),
|
||||
],
|
||||
"pablo_picasso_1881_aa_v4_holdout": [
|
||||
event("picasso_llotja_1895", "education", "1895", "year", "Entered the School of Fine Arts in Barcelona.", BRIT, "https://www.britannica.com/biography/Pablo-Picasso"),
|
||||
event("picasso_moved_paris_1904", "relocation", "1904", "year", "Settled permanently in Paris.", BRIT, "https://www.britannica.com/biography/Pablo-Picasso/Blue-Period"),
|
||||
event("picasso_demoiselles_1907", "career", "1907", "year", "Painted Les Demoiselles d'Avignon.", "Museum of Modern Art", "https://www.moma.org/collection/works/79766"),
|
||||
event("picasso_marriage_1918", "relationship", "1918-07-12", "day", "Married Olga Khokhlova.", WIKI, "https://en.wikipedia.org/wiki/Pablo_Picasso"),
|
||||
event("picasso_paulo_1921", "family", "1921-02-04", "day", "Son Paulo was born.", WIKI, "https://en.wikipedia.org/wiki/Paulo_Picasso"),
|
||||
event("picasso_guernica_1937", "career", "1937", "year", "Created and exhibited Guernica.", "Museum of Modern Art", "https://www.moma.org/collection/works/79018"),
|
||||
event("picasso_jacqueline_1961", "relationship", "1961-03-02", "day", "Married Jacqueline Roque.", WIKI, "https://en.wikipedia.org/wiki/Pablo_Picasso"),
|
||||
],
|
||||
"sigmund_freud_1856_aa_v4_holdout": [
|
||||
event("freud_vienna_md_1881", "education", "1881", "year", "Received his medical degree from the University of Vienna.", BRIT, "https://www.britannica.com/biography/Sigmund-Freud"),
|
||||
event("freud_marriage_1886", "relationship", "1886-09-14", "day", "Married Martha Bernays.", WIKI, "https://en.wikipedia.org/wiki/Sigmund_Freud"),
|
||||
event("freud_anna_1895", "family", "1895-12-03", "day", "Daughter Anna Freud was born.", WIKI, "https://en.wikipedia.org/wiki/Anna_Freud"),
|
||||
event("freud_dreams_publication_1899", "career", "1899", "year", "Published The Interpretation of Dreams.", BRIT, "https://www.britannica.com/biography/Sigmund-Freud/Psychoanalytic-theory"),
|
||||
event("freud_clark_1909", "career", "1909", "year", "Lectured at Clark University in the United States.", WIKI, "https://en.wikipedia.org/wiki/Sigmund_Freud"),
|
||||
event("freud_cancer_1923", "health_pressure", "1923", "year", "Was diagnosed with cancer of the jaw.", BRIT, "https://www.britannica.com/biography/Sigmund-Freud"),
|
||||
event("freud_moved_london_1938", "relocation", "1938", "year", "Fled Nazi-occupied Vienna and settled in London.", WIKI, "https://en.wikipedia.org/wiki/Freud_Museum"),
|
||||
],
|
||||
"salma_hayek_1966_aa_v4_holdout": [
|
||||
event("hayek_iberoamericana_1980s", "education", "1989", "year", "Left Universidad Iberoamericana to act full time.", WIKI, "https://en.wikipedia.org/wiki/Salma_Hayek"),
|
||||
event("hayek_moved_los_angeles_1991", "relocation", "1991", "year", "Moved to Los Angeles to pursue acting in Hollywood.", BRIT, "https://www.britannica.com/biography/Salma-Hayek"),
|
||||
event("hayek_desperado_1995", "career", "1995", "year", "Starred in Desperado.", WIKI, "https://en.wikipedia.org/wiki/Salma_Hayek"),
|
||||
event("hayek_frida_2002", "career", "2002", "year", "Produced and starred in Frida.", OSCARS, "https://www.oscars.org/oscars/ceremonies/2003"),
|
||||
event("hayek_valentina_2007", "family", "2007-09-21", "day", "Daughter Valentina was born.", WIKI, "https://en.wikipedia.org/wiki/Salma_Hayek"),
|
||||
event("hayek_marriage_2009", "relationship", "2009-02-14", "day", "Married Francois-Henri Pinault.", BRIT, "https://www.britannica.com/biography/Salma-Hayek/Personal-life"),
|
||||
event("hayek_oscar_nomination_2003", "career", "2003-03-23", "day", "Received an Academy Award nomination for Frida.", OSCARS, "https://www.oscars.org/oscars/ceremonies/2003"),
|
||||
],
|
||||
"steve_reich_1936_aa_v4_holdout": [
|
||||
event("reich_cornell_degree_1957", "education", "1957", "year", "Graduated from Cornell University.", BRIT, "https://www.britannica.com/biography/Steve-Reich"),
|
||||
event("reich_juilliard_1961", "education", "1961", "year", "Completed studies at the Juilliard School.", WIKI, "https://en.wikipedia.org/wiki/Steve_Reich"),
|
||||
event("reich_moved_california_1963", "relocation", "1963", "year", "Moved to the San Francisco Bay Area.", "Boosey and Hawkes", "https://www.boosey.com/composer/Steve+Reich"),
|
||||
event("reich_return_ny_1965", "relocation", "1965", "year", "Returned to New York.", WIKI, "https://en.wikipedia.org/wiki/Steve_Reich"),
|
||||
event("reich_music_18_premiere_1976", "career", "1976-04-24", "day", "Music for 18 Musicians received its world premiere.", "Pulitzer Prizes", "https://www.pulitzer.org/winners/steve-reich"),
|
||||
event("reich_korot_1976", "relationship", "1976", "year", "Married video artist Beryl Korot.", WIKI, "https://en.wikipedia.org/wiki/Steve_Reich"),
|
||||
event("reich_pulitzer_2009", "career", "2009-04-20", "day", "Won the Pulitzer Prize for Music.", "Pulitzer Prizes", "https://www.pulitzer.org/winners/steve-reich"),
|
||||
],
|
||||
"albert_brooks_1947_aa_v4_holdout": [
|
||||
event("brooks_cmu_1960s", "education", "1966", "year", "Attended Carnegie Mellon University.", WIKI, "https://en.wikipedia.org/wiki/Albert_Brooks"),
|
||||
event("brooks_real_life_1979", "career", "1979", "year", "Made his feature directing debut with Real Life.", WIKI, "https://en.wikipedia.org/wiki/Albert_Brooks"),
|
||||
event("brooks_lost_in_america_1985", "career", "1985", "year", "Wrote and directed Lost in America.", WIKI, "https://en.wikipedia.org/wiki/Lost_in_America"),
|
||||
event("brooks_oscar_nomination_1988", "career", "1988-04-11", "day", "Received an Academy Award nomination for Broadcast News.", OSCARS, "https://www.oscars.org/oscars/ceremonies/1988"),
|
||||
event("brooks_marriage_1997", "relationship", "1997", "year", "Married Kimberly Shlain.", WIKI, "https://en.wikipedia.org/wiki/Kimberly_Brooks_(artist)"),
|
||||
event("brooks_jacob_1998", "family", "1998", "year", "Son Jacob was born.", WIKI, "https://en.wikipedia.org/wiki/Albert_Brooks"),
|
||||
event("brooks_finding_nemo_2003", "career", "2003", "year", "Voiced Marlin in Finding Nemo.", WIKI, "https://en.wikipedia.org/wiki/Albert_Brooks"),
|
||||
],
|
||||
"paul_ryan_1970_aa_v4_holdout": [
|
||||
event("ryan_miami_1992", "education", "1992", "year", "Graduated from Miami University.", BRIT, "https://www.britannica.com/biography/Paul-Ryan"),
|
||||
event("ryan_elected_house_1998", "career", "1998-11-03", "day", "Was elected to the United States House of Representatives.", "United States House of Representatives", "https://history.house.gov/People/Detail/20147"),
|
||||
event("ryan_marriage_2000", "relationship", "2000-12-02", "day", "Married Janna Little.", BRIT, "https://www.britannica.com/biography/Paul-Ryan"),
|
||||
event("ryan_liza_2004", "family", "2004", "year", "Daughter Liza was born.", WIKI, "https://en.wikipedia.org/wiki/Paul_Ryan"),
|
||||
event("ryan_vp_nominee_2012", "career", "2012-08-11", "day", "Was named the Republican vice-presidential nominee.", WIKI, "https://en.wikipedia.org/wiki/Paul_Ryan"),
|
||||
event("ryan_speaker_2015", "career", "2015-10-29", "day", "Was elected Speaker of the House.", WIKI, "https://en.wikipedia.org/wiki/October_2015_Speaker_of_the_United_States_House_of_Representatives_election"),
|
||||
event("ryan_retire_2018", "career", "2018-04-11", "day", "Announced he would not seek re-election.", WIKI, "https://en.wikipedia.org/wiki/Paul_Ryan"),
|
||||
],
|
||||
"tom_kennedy_1927_aa_v4_holdout": [
|
||||
event("kennedy_missouri_1940s", "education", "1947", "year", "Attended the University of Missouri.", WIKI, "https://en.wikipedia.org/wiki/Tom_Kennedy_(television_host)"),
|
||||
event("kennedy_moved_la_1950s", "relocation", "1950", "year", "Moved into network television in Los Angeles.", "The Hollywood Reporter", "https://www.hollywoodreporter.com/tv/tv-news/tom-kennedy-dead-game-show-host-name-that-tune-was-93-4074025/"),
|
||||
event("kennedy_marriage_1952", "relationship", "1952", "year", "Married Betty Gevedon.", "The Hollywood Reporter", "https://www.hollywoodreporter.com/tv/tv-news/tom-kennedy-dead-game-show-host-name-that-tune-was-93-4074025/"),
|
||||
event("kennedy_big_game_1958", "career", "1958", "year", "Began hosting his first national game show, The Big Game.", WIKI, "https://en.wikipedia.org/wiki/Tom_Kennedy_(television_host)"),
|
||||
event("kennedy_you_dont_say_1963", "career", "1963", "year", "Began hosting You Don't Say!", WIKI, "https://en.wikipedia.org/wiki/You_Don%27t_Say!"),
|
||||
event("kennedy_name_that_tune_1974", "career", "1974", "year", "Began hosting the syndicated Name That Tune.", "The Hollywood Reporter", "https://www.hollywoodreporter.com/tv/tv-news/tom-kennedy-dead-game-show-host-name-that-tune-was-93-4074025/"),
|
||||
event("kennedy_death_2020", "health_pressure", "2020-10-07", "day", "Died in a care facility in Oxnard, California.", WIKI, "https://en.wikipedia.org/wiki/Tom_Kennedy_(television_host)"),
|
||||
],
|
||||
"andrew_windsor_1960_aa_v4_holdout": [
|
||||
event("andrew_gordonstoun_1970s", "education", "1979", "year", "Completed schooling at Gordonstoun.", BRIT, "https://www.britannica.com/biography/Prince-Andrew-duke-of-York"),
|
||||
event("andrew_royal_navy_1979", "career", "1979", "year", "Entered the Royal Navy as an officer trainee.", "The Royal Family", "https://www.royal.uk/the-duke-of-york"),
|
||||
event("andrew_falklands_1982", "career", "1982", "year", "Served in the Falklands War as a helicopter pilot.", WIKI, "https://en.wikipedia.org/wiki/Prince_Andrew,_Duke_of_York"),
|
||||
event("andrew_marriage_1986", "relationship", "1986-07-23", "day", "Married Sarah Ferguson.", BRIT, "https://www.britannica.com/biography/Prince-Andrew-duke-of-York"),
|
||||
event("andrew_beatrice_1988", "family", "1988-08-08", "day", "Daughter Beatrice was born.", WIKI, "https://en.wikipedia.org/wiki/Princess_Beatrice"),
|
||||
event("andrew_eugenie_1990", "family", "1990-03-23", "day", "Daughter Eugenie was born.", WIKI, "https://en.wikipedia.org/wiki/Princess_Eugenie"),
|
||||
event("andrew_divorce_1996", "relationship", "1996-05-30", "day", "Divorce from Sarah Ferguson became final.", "The Royal Family", "https://www.royal.uk/duke-york"),
|
||||
],
|
||||
"sean_lennon_1975_aa_v4_holdout": [
|
||||
event("lennon_father_death_1980", "family", "1980-12-08", "day", "His father John Lennon was murdered.", BRIT, "https://www.britannica.com/biography/John-Lennon"),
|
||||
event("lennon_rosey_1980s", "education", "1986", "year", "Attended Institut Le Rosey in Switzerland.", WIKI, "https://en.wikipedia.org/wiki/Sean_Lennon"),
|
||||
event("lennon_columbia_1990s", "education", "1995", "year", "Attended Columbia University.", WIKI, "https://en.wikipedia.org/wiki/Sean_Lennon"),
|
||||
event("lennon_into_sun_1998", "career", "1998-05-19", "day", "Released his debut album Into the Sun.", "AllMusic", "https://www.allmusic.com/album/into-the-sun-mw0000036772"),
|
||||
event("lennon_friendly_fire_2006", "career", "2006-09-26", "day", "Released the album Friendly Fire.", WIKI, "https://en.wikipedia.org/wiki/Friendly_Fire_(Sean_Lennon_album)"),
|
||||
event("lennon_ghost_2012", "career", "2012", "year", "Formed the duo The Ghost of a Saber Tooth Tiger.", WIKI, "https://en.wikipedia.org/wiki/Sean_Lennon"),
|
||||
event("lennon_kemp_muhl", "relationship", "2007", "year", "Began a long-term relationship with Charlotte Kemp Muhl.", WIKI, "https://en.wikipedia.org/wiki/Sean_Lennon"),
|
||||
],
|
||||
"joseph_kennedy_iii_1980_aa_v4_holdout": [
|
||||
event("kennedy_stanford_2003", "education", "2003", "year", "Graduated from Stanford University.", "United States Congress", "https://bioguide.congress.gov/search/bio/K000379"),
|
||||
event("kennedy_peace_corps_2004", "relocation", "2004", "year", "Joined the Peace Corps in the Dominican Republic.", WIKI, "https://en.wikipedia.org/wiki/Joe_Kennedy_III"),
|
||||
event("kennedy_harvard_law_2009", "education", "2009", "year", "Graduated from Harvard Law School.", WIKI, "https://en.wikipedia.org/wiki/Joe_Kennedy_III"),
|
||||
event("kennedy_elected_house_2012", "career", "2012-11-06", "day", "Was elected to the United States House of Representatives.", WIKI, "https://en.wikipedia.org/wiki/2012_United_States_House_of_Representatives_elections_in_Massachusetts"),
|
||||
event("kennedy_marriage_2012", "relationship", "2012-12-01", "day", "Married Lauren Birchfield.", WIKI, "https://en.wikipedia.org/wiki/Joe_Kennedy_III"),
|
||||
event("kennedy_daughter_2015", "family", "2015", "year", "Their first child was born.", WIKI, "https://en.wikipedia.org/wiki/Joe_Kennedy_III"),
|
||||
event("kennedy_senate_primary_2020", "career", "2020-09-01", "day", "Lost the Democratic Senate primary in Massachusetts.", WIKI, "https://en.wikipedia.org/wiki/2020_United_States_Senate_election_in_Massachusetts"),
|
||||
],
|
||||
"kurt_cobain_1967_aa_v4_holdout": [
|
||||
event("cobain_left_school_1985", "education", "1985", "year", "Dropped out of high school shortly before graduation.", WIKI, "https://en.wikipedia.org/wiki/Kurt_Cobain"),
|
||||
event("cobain_olympia_1987", "relocation", "1987", "year", "Moved to Olympia, Washington.", WIKI, "https://en.wikipedia.org/wiki/Kurt_Cobain"),
|
||||
event("cobain_bleach_1989", "career", "1989-06-15", "day", "Nirvana released Bleach.", WIKI, "https://en.wikipedia.org/wiki/Bleach_(Nirvana_album)"),
|
||||
event("cobain_nevermind_1991", "career", "1991-09-24", "day", "Nirvana released Nevermind.", "Rock and Roll Hall of Fame", "https://www.rockhall.com/inductees/nirvana"),
|
||||
event("cobain_marriage_1992", "relationship", "1992-02-24", "day", "Married Courtney Love.", BRIT, "https://www.britannica.com/biography/Kurt-Cobain"),
|
||||
event("cobain_frances_1992", "family", "1992-08-18", "day", "Daughter Frances Bean was born.", WIKI, "https://en.wikipedia.org/wiki/Frances_Bean_Cobain"),
|
||||
event("cobain_overdose_1994", "health_pressure", "1994-03-04", "day", "Was hospitalized after an overdose in Rome.", "The Seattle Times", "https://www.seattletimes.com/entertainment/music/kurt-cobain-a-timeline-of-his-life/"),
|
||||
],
|
||||
"john_robbins_1947_aa_v4_holdout": [
|
||||
event("robbins_marriage_1968", "relationship", "1967-03-10", "day", "Married Deo Robbins.", WIKI, "https://en.wikipedia.org/wiki/John_Robbins_(author)"),
|
||||
event("robbins_berkeley_1969", "education", "1969", "year", "Graduated from the University of California, Berkeley.", WIKI, "https://en.wikipedia.org/wiki/John_Robbins_(author)"),
|
||||
event("robbins_antioch_1976", "education", "1976", "year", "Received a master's degree from Antioch College.", WIKI, "https://en.wikipedia.org/wiki/John_Robbins_(author)"),
|
||||
event("robbins_diet_book_1987", "career", "1987", "year", "Published Diet for a New America.", WIKI, "https://en.wikipedia.org/wiki/John_Robbins_(author)"),
|
||||
event("robbins_earthsave_1988", "career", "1988", "year", "Founded the organization EarthSave.", WIKI, "https://en.wikipedia.org/wiki/John_Robbins_(author)"),
|
||||
event("robbins_health_book_1996", "career", "1996", "year", "Published Reclaiming Our Health.", "John Robbins", "https://www.johnrobbins.info/books/"),
|
||||
event("robbins_death_2025", "health_pressure", "2025-06-11", "day", "Died in Soquel, California.", WIKI, "https://en.wikipedia.org/wiki/John_Robbins_(author)"),
|
||||
],
|
||||
"chad_everett_1937_aa_v4_holdout": [
|
||||
event("everett_wayne_state_1950s", "education", "1957", "year", "Attended Wayne State University.", WIKI, "https://en.wikipedia.org/wiki/Chad_Everett"),
|
||||
event("everett_moved_hollywood_1960", "relocation", "1960", "year", "Moved to Hollywood under a studio contract.", WIKI, "https://en.wikipedia.org/wiki/Chad_Everett"),
|
||||
event("everett_marriage_1966", "relationship", "1966-05-22", "day", "Married actress Shelby Grant.", WIKI, "https://en.wikipedia.org/wiki/Chad_Everett"),
|
||||
event("everett_katherine_1967", "family", "1967", "year", "Daughter Katherine was born.", WIKI, "https://en.wikipedia.org/wiki/Chad_Everett"),
|
||||
event("everett_medical_center_1969", "career", "1969", "year", "Began starring in Medical Center.", WIKI, "https://en.wikipedia.org/wiki/Medical_Center_(TV_series)"),
|
||||
event("everett_airplane_ii_1982", "career", "1982", "year", "Appeared in Airplane II: The Sequel.", WIKI, "https://en.wikipedia.org/wiki/Chad_Everett"),
|
||||
event("everett_lung_cancer_2012", "health_pressure", "2012", "year", "Died after a battle with lung cancer.", "The New York Times", "https://www.nytimes.com/2012/07/26/arts/television/chad-everett-medical-center-star-dies-at-75.html"),
|
||||
],
|
||||
"bernd_eichinger_1949_aa_v4_holdout": [
|
||||
event("eichinger_hff_1970s", "education", "1973", "year", "Studied at the University of Television and Film Munich.", WIKI, "https://en.wikipedia.org/wiki/Bernd_Eichinger"),
|
||||
event("eichinger_constantin_1979", "career", "1979", "year", "Bought a stake in Neue Constantin Film and became executive director.", WIKI, "https://en.wikipedia.org/wiki/Constantin_Film"),
|
||||
event("eichinger_name_of_rose_1986", "career", "1986", "year", "Produced The Name of the Rose.", WIKI, "https://en.wikipedia.org/wiki/The_Name_of_the_Rose_(film)"),
|
||||
event("eichinger_summit_1991", "career", "1991", "year", "Co-founded Summit Entertainment.", WIKI, "https://en.wikipedia.org/wiki/Summit_Entertainment"),
|
||||
event("eichinger_nina_1994", "family", "1994", "year", "Daughter Nina was born.", WIKI, "https://en.wikipedia.org/wiki/Bernd_Eichinger"),
|
||||
event("eichinger_marriage_2006", "relationship", "2006", "year", "Married Katja Hofmann.", "Reuters", "https://www.reuters.com/article/us-berndeichinger/film-producer-bernd-eichinger-dies-at-61-idUSTRE70O6BJ20110125"),
|
||||
event("eichinger_heart_2011", "health_pressure", "2011-01-24", "day", "Died of a heart attack in Los Angeles.", WIKI, "https://en.wikipedia.org/wiki/Bernd_Eichinger"),
|
||||
],
|
||||
"iwao_takamoto_1925_aa_v4_holdout": [
|
||||
event("takamoto_manzanar_1942", "relocation", "1942", "year", "Was sent with his family to the Manzanar internment camp.", WIKI, "https://en.wikipedia.org/wiki/Iwao_Takamoto"),
|
||||
event("takamoto_joined_disney_1947", "career", "1945", "year", "Joined Walt Disney Animation Studios.", WIKI, "https://en.wikipedia.org/wiki/Iwao_Takamoto"),
|
||||
event("takamoto_cinderella_1950", "career", "1950", "year", "Worked as a character artist on Cinderella.", WIKI, "https://en.wikipedia.org/wiki/Iwao_Takamoto"),
|
||||
event("takamoto_marriage_shattuck_1957", "relationship", "1957", "year", "Married Jane M. Shattuck.", WIKI, "https://en.wikipedia.org/wiki/Iwao_Takamoto"),
|
||||
event("takamoto_hanna_barbera_1961", "career", "1961", "year", "Moved to Hanna-Barbera and began designing its characters.", "Animation Magazine", "https://www.animationmagazine.net/2007/01/animation-legend-iwao-takamoto-dies/"),
|
||||
event("takamoto_marriage_farber_1964", "relationship", "1964", "year", "Married Barbara Farber.", WIKI, "https://en.wikipedia.org/wiki/Iwao_Takamoto"),
|
||||
event("takamoto_scooby_1969", "career", "1969", "year", "Scooby-Doo, Where Are You! premiered with his character design.", WIKI, "https://en.wikipedia.org/wiki/Scooby-Doo,_Where_Are_You!"),
|
||||
event("takamoto_heart_attack_2007", "health_pressure", "2007-01-08", "day", "Died following a heart attack.", "The New York Times", "https://www.nytimes.com/2007/01/10/arts/10takamoto.html"),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _validate(case: dict[str, Any]) -> None:
|
||||
events = case["events"]
|
||||
domains = {str(item["domain"]) for item in events}
|
||||
if len(events) < MIN_EVENTS:
|
||||
raise SystemExit(f"{case['case_id']}: {len(events)} events < {MIN_EVENTS}")
|
||||
if len(domains) < MIN_DOMAINS:
|
||||
raise SystemExit(f"{case['case_id']}: {sorted(domains)} domains < {MIN_DOMAINS}")
|
||||
rating = ((case.get("birth") or {}).get("source") or {}).get("rodden_rating")
|
||||
if rating != "AA":
|
||||
raise SystemExit(f"{case['case_id']}: rodden_rating {rating!r} is not AA")
|
||||
|
||||
|
||||
def build() -> dict[str, Any]:
|
||||
v3 = json.loads(V3_PATH.read_text(encoding="utf-8"))
|
||||
cases = []
|
||||
for row in v3["cases"]:
|
||||
case_id = str(row["case_id"])
|
||||
extra = EVENTS.get(case_id)
|
||||
if extra is None:
|
||||
raise SystemExit(f"missing v4 events for {case_id}")
|
||||
case = {
|
||||
**row,
|
||||
"holdout_partition": "open_scale_evaluation",
|
||||
"excluded_from_tuning": True,
|
||||
"candidate_radius_minutes": 10,
|
||||
"candidate_radii_minutes": list(RADII),
|
||||
"true_minute": str(row["birth"]["time"])[:5],
|
||||
"true_cluster_note": "Cluster membership depends on the scoring variant and radius; recorded in the sweep results.",
|
||||
"events": extra,
|
||||
}
|
||||
_validate(case)
|
||||
cases.append(case)
|
||||
return {
|
||||
"schema_version": "minute-rectification-holdout-v4",
|
||||
"benchmark_id": "minute_rectification_scale_holdout_v4",
|
||||
"description": "Public Rodden-AA minute set with at least 7 dated events and 4 domains per case, scored at ±10 / ±30 / ±60. Open evaluation for scoring-scale research, not a sealed blind release gate.",
|
||||
"truth_hidden_from_ranker": False,
|
||||
"frozen_before_replay": False,
|
||||
"source_audit_status": "public_biography_expansion_2026_09_14",
|
||||
"ayanamsa": "raman",
|
||||
"node_mode": "mean",
|
||||
"candidate_radii_minutes": list(RADII),
|
||||
"minimum_gate": {
|
||||
"public_aa_cases": len(cases),
|
||||
"events_per_case": MIN_EVENTS,
|
||||
"domains_per_case": MIN_DOMAINS,
|
||||
},
|
||||
"v3_date_corrections": [
|
||||
{"event_id": "robbins_marriage_1968", "submitted": "1968", "working": "1967-03-10"},
|
||||
{"event_id": "takamoto_joined_disney_1947", "submitted": "1947", "working": "1945"},
|
||||
],
|
||||
"parent_dataset": "references/real_case_calibration/minute_rectification_holdout_v3.json",
|
||||
"cases": cases,
|
||||
"boundary": "Not a sealed blind holdout. Events are public biographies. Do not mix these numbers with upstream true-node runs.",
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
payload = build()
|
||||
V4_PATH.write_text(json.dumps(payload, ensure_ascii=True, indent=2) + "\n", encoding="utf-8")
|
||||
print(f"wrote {V4_PATH} cases={len(payload['cases'])}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,346 @@
|
||||
"""Pure helpers for the minute-resolution scoring sweep. No production defaults."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from collections.abc import Iterable, Sequence
|
||||
from typing import Any
|
||||
|
||||
from scripts.active_rectification_event_engine import DOMAIN_CONFIG
|
||||
from scripts.rectification.candidate_contrast import (
|
||||
SIGNATURE_LAYERS,
|
||||
cluster_contexts_by_signature,
|
||||
context_time,
|
||||
layer_value,
|
||||
select_signature_representatives,
|
||||
)
|
||||
from scripts.rectification.scoring_service import PRECISION_WEIGHTS, _event_kind_factor
|
||||
|
||||
VARGA_RULE_WEIGHT = {
|
||||
"vim_md_domain_varga": 2.0,
|
||||
"vim_ad_domain_varga": 1.5,
|
||||
"vim_pd_domain_varga": 0.75,
|
||||
}
|
||||
|
||||
KP_MATCH = {
|
||||
"md": (0.50, 0.25),
|
||||
"ad": (0.35, 0.15),
|
||||
"pd": (0.20, 0.10),
|
||||
}
|
||||
|
||||
DOMAIN_SIGNATURE_LAYERS = {
|
||||
"education": ("d24", "d5"),
|
||||
"relocation": ("d4",),
|
||||
"relationship": ("d9",),
|
||||
"career": ("d10",),
|
||||
"occupation": ("d10",),
|
||||
"finance": ("d2", "d11"),
|
||||
"health_pressure": ("d30",),
|
||||
"family": ("d12", "d7", "d3"),
|
||||
}
|
||||
|
||||
EXTRA_LAYER_VARGA = {
|
||||
"d2": "D2",
|
||||
"d3": "D3",
|
||||
"d5": "D5",
|
||||
"d7": "D7",
|
||||
"d11": "D11",
|
||||
"d30": "D30",
|
||||
}
|
||||
|
||||
NARROW_WIDTH = 5
|
||||
|
||||
|
||||
def clock(value: str) -> int:
|
||||
stamp = str(value)[:5]
|
||||
return int(stamp[:2]) * 60 + int(stamp[3:5])
|
||||
|
||||
|
||||
def hhmm_from_minutes(value: int) -> str:
|
||||
wrapped = value % 1440
|
||||
return f"{wrapped // 60:02d}:{wrapped % 60:02d}"
|
||||
|
||||
|
||||
def shift_clock(value: str, delta: int) -> str:
|
||||
return hhmm_from_minutes(clock(value) + delta)
|
||||
|
||||
|
||||
def range_width(times: Sequence[str]) -> int | None:
|
||||
clocks = sorted(clock(item) for item in times if str(item)[:5])
|
||||
if not clocks:
|
||||
return None
|
||||
return clocks[-1] - clocks[0] + 1
|
||||
|
||||
|
||||
def varga_divisor(count: int, mode: str) -> float:
|
||||
n = max(int(count), 1)
|
||||
if mode == "len":
|
||||
return float(n)
|
||||
if mode == "sqrt":
|
||||
return math.sqrt(n)
|
||||
if mode == "fixed2":
|
||||
return 2.0
|
||||
return 2.0 * n
|
||||
|
||||
|
||||
def rescale_varga_points(
|
||||
points: float,
|
||||
rule_ids: Sequence[str],
|
||||
domain: str,
|
||||
event_kind: str,
|
||||
precision: str,
|
||||
mode: str,
|
||||
) -> float:
|
||||
if mode == "2len":
|
||||
return float(points)
|
||||
prefixes, _houses = DOMAIN_CONFIG[domain]
|
||||
n = max(len(prefixes), 1)
|
||||
old_div = varga_divisor(n, "2len")
|
||||
new_div = varga_divisor(n, mode)
|
||||
if abs(old_div - new_div) < 1e-12:
|
||||
return float(points)
|
||||
delta_inner = 0.0
|
||||
for rule in rule_ids:
|
||||
weight = VARGA_RULE_WEIGHT.get(str(rule))
|
||||
if weight is None:
|
||||
continue
|
||||
delta_inner += weight * (1.0 / new_div - 1.0 / old_div)
|
||||
precision_w = PRECISION_WEIGHTS.get(precision, 1.0)
|
||||
kind = _event_kind_factor(event_kind, rule_ids)
|
||||
return round(float(points) + delta_inner * kind * (precision_w ** 2), 4)
|
||||
|
||||
|
||||
def aggregate_samples(samples: Sequence[float], mode: str, temperature: float = 1.0) -> float:
|
||||
values = [float(item) for item in samples]
|
||||
if not values:
|
||||
return 0.0
|
||||
if mode == "max":
|
||||
return round(max(values), 4)
|
||||
if mode == "lse":
|
||||
temp = max(float(temperature), 1e-6)
|
||||
peak = max(values)
|
||||
mean_exp = sum(math.exp((item - peak) / temp) for item in values) / len(values)
|
||||
return round(temp * math.log(mean_exp) + peak, 4)
|
||||
return round(sum(values) / len(values), 4)
|
||||
|
||||
|
||||
def subtract_event_floor(by_time: dict[str, float]) -> dict[str, float]:
|
||||
if not by_time:
|
||||
return {}
|
||||
floor = min(by_time.values())
|
||||
return {time: round(score - floor, 4) for time, score in by_time.items()}
|
||||
|
||||
|
||||
def shannon_entropy(weights: Iterable[float]) -> float:
|
||||
values = [max(float(item), 0.0) for item in weights]
|
||||
total = sum(values)
|
||||
if total <= 0:
|
||||
return 0.0
|
||||
entropy = 0.0
|
||||
for value in values:
|
||||
if value <= 0:
|
||||
continue
|
||||
share = value / total
|
||||
entropy -= share * math.log(share, 2)
|
||||
return round(entropy, 6)
|
||||
|
||||
|
||||
def kp_event_points(
|
||||
context: dict[str, Any],
|
||||
domain: str,
|
||||
vim_lords: tuple[str, str, str],
|
||||
weight: float,
|
||||
) -> float:
|
||||
feature = context.get("feature") if isinstance(context.get("feature"), dict) else {}
|
||||
snapshot = feature.get("kp_cusps") if isinstance(feature.get("kp_cusps"), dict) else {}
|
||||
if snapshot.get("status") != "executed":
|
||||
return 0.0
|
||||
houses = snapshot.get("houses") if isinstance(snapshot.get("houses"), dict) else {}
|
||||
_prefixes, target_houses = DOMAIN_CONFIG[domain]
|
||||
points = 0.0
|
||||
md, ad, pd = vim_lords
|
||||
for house in target_houses:
|
||||
row = houses.get(str(house))
|
||||
if not isinstance(row, dict):
|
||||
continue
|
||||
sub = str(row.get("sub_lord") or "")
|
||||
sub_sub = str(row.get("sub_sub_lord") or "")
|
||||
for lord, (sub_w, sub_sub_w) in (
|
||||
(md, KP_MATCH["md"]),
|
||||
(ad, KP_MATCH["ad"]),
|
||||
(pd, KP_MATCH["pd"]),
|
||||
):
|
||||
if lord and lord == sub:
|
||||
points += sub_w
|
||||
if lord and lord == sub_sub:
|
||||
points += sub_sub_w
|
||||
return round(points * float(weight), 4)
|
||||
|
||||
|
||||
def kp_sub_lord_key(context: dict[str, Any], houses: Sequence[int]) -> tuple[str, ...]:
|
||||
feature = context.get("feature") if isinstance(context.get("feature"), dict) else {}
|
||||
snapshot = feature.get("kp_cusps") if isinstance(feature.get("kp_cusps"), dict) else {}
|
||||
table = snapshot.get("houses") if isinstance(snapshot.get("houses"), dict) else {}
|
||||
keys = []
|
||||
for house in houses:
|
||||
row = table.get(str(house)) if isinstance(table.get(str(house)), dict) else {}
|
||||
keys.append(str(row.get("sub_lord") or ""))
|
||||
return tuple(keys)
|
||||
|
||||
|
||||
def kp_changes_in_window(contexts: Sequence[dict[str, Any]], domain: str) -> int:
|
||||
_prefixes, houses = DOMAIN_CONFIG[domain]
|
||||
seen: list[tuple[str, ...]] = []
|
||||
for context in contexts:
|
||||
key = kp_sub_lord_key(context, houses)
|
||||
if not seen or seen[-1] != key:
|
||||
seen.append(key)
|
||||
return max(len(seen) - 1, 0)
|
||||
|
||||
|
||||
def dynamic_signature_layers(domains: Sequence[str]) -> tuple[str, ...]:
|
||||
layers = ["d1"]
|
||||
for domain in domains:
|
||||
for layer in DOMAIN_SIGNATURE_LAYERS.get(domain, ()):
|
||||
if layer not in layers:
|
||||
layers.append(layer)
|
||||
return tuple(layers)
|
||||
|
||||
|
||||
def layer_value_extended(context: dict[str, Any], layer: str) -> int | None:
|
||||
if layer in SIGNATURE_LAYERS or layer == "md" or layer == "d1":
|
||||
return layer_value(context, layer)
|
||||
name = EXTRA_LAYER_VARGA.get(layer)
|
||||
if not name:
|
||||
return None
|
||||
feature = context.get("feature") if isinstance(context.get("feature"), dict) else {}
|
||||
vargas = feature.get("varga_ascendants") if isinstance(feature.get("varga_ascendants"), dict) else {}
|
||||
raw = vargas.get(name)
|
||||
return raw if isinstance(raw, int) else None
|
||||
|
||||
|
||||
def cluster_contexts(
|
||||
contexts: Sequence[dict[str, Any]],
|
||||
layers: Sequence[str] | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
if not layers or tuple(layers) == SIGNATURE_LAYERS:
|
||||
return cluster_contexts_by_signature(contexts)
|
||||
buckets: dict[tuple[int | None, ...], list[dict[str, Any]]] = {}
|
||||
for context in contexts:
|
||||
if not isinstance(context, dict):
|
||||
continue
|
||||
time = context_time(context)
|
||||
if not time:
|
||||
continue
|
||||
signature = tuple(layer_value_extended(context, layer) for layer in layers)
|
||||
buckets.setdefault(signature, []).append(context)
|
||||
clusters: list[dict[str, Any]] = []
|
||||
for signature, members in buckets.items():
|
||||
ordered = sorted(members, key=lambda item: clock(str(context_time(item))))
|
||||
times = [str(context_time(item)) for item in ordered]
|
||||
clusters.append({
|
||||
"signature": signature,
|
||||
"signature_key": ",".join("x" if value is None else str(value) for value in signature),
|
||||
"contexts": ordered,
|
||||
"times": times,
|
||||
"representative_time": times[len(times) // 2],
|
||||
"representative": ordered[len(ordered) // 2],
|
||||
})
|
||||
clusters.sort(key=lambda item: clock(item["representative_time"]))
|
||||
return clusters
|
||||
|
||||
|
||||
def public_rows(
|
||||
rows: Sequence[dict[str, Any]],
|
||||
contexts: Sequence[dict[str, Any]],
|
||||
layers: Sequence[str] | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
if not layers or tuple(layers) == SIGNATURE_LAYERS:
|
||||
return select_signature_representatives(rows, contexts)
|
||||
by_time = {str(row.get("time"))[:5]: row for row in rows if str(row.get("time"))}
|
||||
clusters = cluster_contexts(contexts, layers)
|
||||
from scripts.rectification.candidate_contrast import cap_clusters_by_adjacent_merge
|
||||
clusters = cap_clusters_by_adjacent_merge(clusters, by_time)
|
||||
representatives: list[dict[str, Any]] = []
|
||||
for cluster in clusters:
|
||||
members = [by_time[time] for time in cluster["times"] if time in by_time]
|
||||
if not members:
|
||||
continue
|
||||
best = max(members, key=lambda row: (float(row.get("score") or 0), str(row.get("time"))))
|
||||
representatives.append({
|
||||
**best,
|
||||
"cluster_times": [time for time in cluster["times"] if time in by_time],
|
||||
})
|
||||
representatives.sort(key=lambda row: (-float(row.get("score") or 0), str(row.get("time"))))
|
||||
return representatives or list(rows)[:1]
|
||||
|
||||
|
||||
def cluster_of(true_time: str, clusters: Sequence[dict[str, Any]]) -> dict[str, Any] | None:
|
||||
stamp = str(true_time)[:5]
|
||||
for cluster in clusters:
|
||||
times = [str(item)[:5] for item in cluster.get("times") or []]
|
||||
if stamp in times:
|
||||
return cluster
|
||||
return None
|
||||
|
||||
|
||||
def metrics_from_public(
|
||||
public: Sequence[dict[str, Any]],
|
||||
true_time: str,
|
||||
window_times: Sequence[str],
|
||||
) -> dict[str, Any]:
|
||||
if not public:
|
||||
return {
|
||||
"top1_hit": False,
|
||||
"coverage": False,
|
||||
"width": None,
|
||||
"tie": False,
|
||||
"entropy": 0.0,
|
||||
"truth_squeezed": True,
|
||||
"too_narrow": False,
|
||||
"leader_count": 0,
|
||||
"public_count": 0,
|
||||
"true_cluster_rank": None,
|
||||
"true_cluster_size": 0,
|
||||
}
|
||||
scores = [float(row.get("score") or 0) for row in public]
|
||||
best = max(scores)
|
||||
leaders = [row for row in public if abs(float(row.get("score") or 0) - best) <= 1e-9]
|
||||
delivered = []
|
||||
seen: set[str] = set()
|
||||
for row in public:
|
||||
for time in row.get("cluster_times") or [str(row.get("time"))[:5]]:
|
||||
stamp = str(time)[:5]
|
||||
if stamp not in seen:
|
||||
seen.add(stamp)
|
||||
delivered.append(stamp)
|
||||
true = str(true_time)[:5]
|
||||
true_in_leaders = any(true in (row.get("cluster_times") or [str(row.get("time"))[:5]]) for row in leaders)
|
||||
if not true_in_leaders:
|
||||
true_in_leaders = any(str(row.get("time"))[:5] == true for row in leaders)
|
||||
rank = None
|
||||
size = 0
|
||||
ordered = sorted(public, key=lambda row: (-float(row.get("score") or 0), str(row.get("time"))))
|
||||
for index, row in enumerate(ordered, start=1):
|
||||
members = [str(item)[:5] for item in (row.get("cluster_times") or [str(row.get("time"))[:5]])]
|
||||
if true in members:
|
||||
rank = index
|
||||
size = len(members)
|
||||
break
|
||||
width = range_width(delivered)
|
||||
second = sorted({round(float(row.get("score") or 0), 4) for row in public}, reverse=True)
|
||||
tied = len(leaders) >= 2 or (len(second) >= 2 and abs(second[0] - second[1]) <= 1e-9)
|
||||
return {
|
||||
"top1_hit": true_in_leaders,
|
||||
"coverage": true in seen,
|
||||
"width": width,
|
||||
"tie": tied,
|
||||
"entropy": shannon_entropy(max(score, 0.0) for score in scores),
|
||||
"truth_squeezed": true not in seen,
|
||||
"too_narrow": width is not None and width <= NARROW_WIDTH and true in seen,
|
||||
"leader_count": len(leaders),
|
||||
"public_count": len(public),
|
||||
"true_cluster_rank": rank,
|
||||
"true_cluster_size": size,
|
||||
"window_width": range_width(list(window_times)),
|
||||
}
|
||||
@@ -0,0 +1,599 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Offline sweep of minute-resolution scoring changes.
|
||||
|
||||
Reads production scoring modules. Does not change their defaults.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import statistics
|
||||
import sys
|
||||
import traceback
|
||||
from collections import defaultdict
|
||||
from dataclasses import dataclass
|
||||
from datetime import date, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Sequence
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts.active_rectification_event_engine import ( # noqa: E402
|
||||
AYANAMSA,
|
||||
DOMAIN_CONFIG,
|
||||
NODE_MODE,
|
||||
_active_vimshottari,
|
||||
compute_candidate_static_contexts,
|
||||
compute_event_candidate_rows,
|
||||
)
|
||||
from scripts.rectification.event_probes import discriminating_event_probes # noqa: E402
|
||||
from scripts.rectification.refinement_packet import window_scan # noqa: E402
|
||||
from scripts.rectification.scoring_service import ( # noqa: E402
|
||||
_kind_adjusted_evidence,
|
||||
_legacy_request,
|
||||
sample_event_dates,
|
||||
score_from_matrix,
|
||||
scoreable_request,
|
||||
)
|
||||
from scripts.research.minute_resolution_lib import ( # noqa: E402
|
||||
SIGNATURE_LAYERS,
|
||||
aggregate_samples,
|
||||
cluster_contexts,
|
||||
cluster_of,
|
||||
dynamic_signature_layers,
|
||||
kp_changes_in_window,
|
||||
kp_event_points,
|
||||
metrics_from_public,
|
||||
public_rows,
|
||||
rescale_varga_points,
|
||||
shannon_entropy,
|
||||
subtract_event_floor,
|
||||
)
|
||||
from scripts.research.probe_supply_after_six import ( # noqa: E402
|
||||
ASK_COUNT,
|
||||
apply_answer,
|
||||
optimal_answer,
|
||||
remaining_after_six,
|
||||
request_from_case,
|
||||
top1_hit,
|
||||
)
|
||||
|
||||
HOLDOUT = ROOT / "references" / "real_case_calibration" / "minute_rectification_holdout_v4.json"
|
||||
REPORT_MD = ROOT / "docs" / "research" / "minute_resolution_2026_09_14.md"
|
||||
REPORT_JSON = ROOT / "docs" / "research" / "minute_resolution_2026_09_14.json"
|
||||
TODAY = date(2026, 9, 14)
|
||||
RADII = (10, 30, 60)
|
||||
MINUTE_STEP = 2
|
||||
ASK = ASK_COUNT
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Variant:
|
||||
name: str
|
||||
r1: str = "2len"
|
||||
r2: bool = False
|
||||
r3: float | None = None
|
||||
r4: str = "mean"
|
||||
r5: bool = False
|
||||
|
||||
@property
|
||||
def uses_r3(self) -> bool:
|
||||
return self.r3 is not None
|
||||
|
||||
|
||||
def all_variants() -> list[Variant]:
|
||||
rows = [
|
||||
Variant("baseline"),
|
||||
Variant("R1@len", r1="len"),
|
||||
Variant("R1@sqrt", r1="sqrt"),
|
||||
Variant("R1@fixed2", r1="fixed2"),
|
||||
Variant("R2", r2=True),
|
||||
Variant("R3@0.5", r3=0.5),
|
||||
Variant("R3@1.0", r3=1.0),
|
||||
Variant("R3@2.0", r3=2.0),
|
||||
Variant("R4@max", r4="max"),
|
||||
Variant("R4@lse", r4="lse"),
|
||||
Variant("R5", r5=True),
|
||||
]
|
||||
keys = ("R1", "R2", "R3", "R4", "R5")
|
||||
for i, left in enumerate(keys):
|
||||
for right in keys[i + 1:]:
|
||||
pair = {left, right}
|
||||
rows.append(Variant(
|
||||
f"{left}+{right}",
|
||||
r1="len" if "R1" in pair else "2len",
|
||||
r2="R2" in pair,
|
||||
r3=(1.0 if "R3" in pair else None),
|
||||
r4="lse" if "R4" in pair else "mean",
|
||||
r5="R5" in pair,
|
||||
))
|
||||
return rows
|
||||
|
||||
|
||||
def scoring_request_for(case: dict[str, Any], radius: int) -> dict[str, Any]:
|
||||
previous = case.get("candidate_radius_minutes")
|
||||
case["candidate_radius_minutes"] = radius
|
||||
try:
|
||||
request = request_from_case(case)
|
||||
finally:
|
||||
if previous is None:
|
||||
case.pop("candidate_radius_minutes", None)
|
||||
else:
|
||||
case["candidate_radius_minutes"] = previous
|
||||
request["minute_step"] = MINUTE_STEP
|
||||
request["ayanamsa"] = AYANAMSA
|
||||
request["node_mode"] = NODE_MODE
|
||||
return scoreable_request(request)
|
||||
|
||||
|
||||
def event_at(sample: str) -> datetime:
|
||||
return datetime.strptime(f"{sample} 12:00:00", "%Y-%m-%d %H:%M:%S")
|
||||
|
||||
|
||||
def collect_samples(
|
||||
request: dict[str, Any],
|
||||
static_contexts: Sequence[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
times = [
|
||||
item["feature"]["time"] if isinstance(item.get("feature"), dict) else item["candidate_at"].strftime("%H:%M")
|
||||
for item in static_contexts
|
||||
]
|
||||
events = list(request["events"])
|
||||
payload: dict[str, dict[str, list[dict[str, Any]]]] = {event["id"]: {time: [] for time in times} for event in events}
|
||||
kp_by_event: dict[str, dict[str, list[float]]] = {event["id"]: {time: [] for time in times} for event in events}
|
||||
domains = [str(event["domain"]) for event in events]
|
||||
for event in events:
|
||||
samples = sample_event_dates(event)
|
||||
for sampled in samples:
|
||||
rows = list(compute_event_candidate_rows(
|
||||
_legacy_request(request, event, sampled),
|
||||
static_contexts=static_contexts,
|
||||
))
|
||||
by_time = {str(row["time"])[:5]: row for row in rows}
|
||||
event_dt = event_at(sampled)
|
||||
for context in static_contexts:
|
||||
time = str(context["feature"]["time"])[:5]
|
||||
row = by_time.get(time)
|
||||
if row is None or not row.get("evidence"):
|
||||
continue
|
||||
adjusted = _kind_adjusted_evidence(event, row["evidence"][0])
|
||||
payload[event["id"]][time].append({
|
||||
"points": float(adjusted["points"]),
|
||||
"rule_ids": list(adjusted["rule_ids"]),
|
||||
})
|
||||
moon = context["planet_longitudes"]["Moon"]
|
||||
try:
|
||||
vim = _active_vimshottari(request["birth_date"], float(moon), event_dt)
|
||||
except (KeyError, TypeError, ValueError):
|
||||
vim = ("", "", "")
|
||||
kp_by_event[event["id"]][time].append(
|
||||
kp_event_points(context, event["domain"], vim, 1.0),
|
||||
)
|
||||
return {
|
||||
"times": times,
|
||||
"events": events,
|
||||
"domains": domains,
|
||||
"samples": payload,
|
||||
"kp_samples": kp_by_event,
|
||||
"static_contexts": list(static_contexts),
|
||||
}
|
||||
|
||||
|
||||
def apply_variant(bundle: dict[str, Any], variant: Variant) -> dict[str, dict[str, dict[str, Any]]]:
|
||||
matrix: dict[str, dict[str, dict[str, Any]]] = {}
|
||||
for event in bundle["events"]:
|
||||
event_id = event["id"]
|
||||
domain = event["domain"]
|
||||
kind = event["event_kind"]
|
||||
precision = event["precision"]
|
||||
by_time: dict[str, float] = {}
|
||||
rules_by_time: dict[str, list[str]] = {}
|
||||
for time, samples in bundle["samples"][event_id].items():
|
||||
if not samples:
|
||||
continue
|
||||
scaled = [
|
||||
rescale_varga_points(item["points"], item["rule_ids"], domain, kind, precision, variant.r1)
|
||||
for item in samples
|
||||
]
|
||||
kp_scaled = [
|
||||
round(value * float(variant.r3), 4)
|
||||
for value in bundle["kp_samples"][event_id].get(time, [])
|
||||
] if variant.r3 is not None else [0.0] * len(scaled)
|
||||
combined = [left + right for left, right in zip(scaled, kp_scaled)] or scaled
|
||||
by_time[time] = aggregate_samples(combined, variant.r4)
|
||||
rules_by_time[time] = sorted({rule for item in samples for rule in item["rule_ids"]})
|
||||
if variant.r2:
|
||||
by_time = subtract_event_floor(by_time)
|
||||
matrix[event_id] = {
|
||||
time: {"points": score, "rule_ids": rules_by_time.get(time, [])}
|
||||
for time, score in by_time.items()
|
||||
}
|
||||
return matrix
|
||||
|
||||
|
||||
def built_payload(bundle: dict[str, Any], matrix: dict[str, dict[str, dict[str, Any]]]) -> dict[str, Any]:
|
||||
return {
|
||||
"candidate_times": list(bundle["times"]),
|
||||
"matrix": matrix,
|
||||
"date_sensitivity": [],
|
||||
"missing_layers": [],
|
||||
"static_contexts": bundle["static_contexts"],
|
||||
}
|
||||
|
||||
|
||||
def entropy_curve(
|
||||
*,
|
||||
request: dict[str, Any],
|
||||
built: dict[str, Any],
|
||||
prior: dict[str, float],
|
||||
true_time: str,
|
||||
clusters: Sequence[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
times = list(prior)
|
||||
initial = discriminating_event_probes(
|
||||
{**request, "refresh_probes": False},
|
||||
built,
|
||||
scan=window_scan(built),
|
||||
candidate_times=times,
|
||||
representative_time=true_time,
|
||||
today=TODAY,
|
||||
)
|
||||
asked = initial[:ASK]
|
||||
scores = dict(prior)
|
||||
conflicts = {time: 0 for time in times}
|
||||
eliminated: set[str] = set()
|
||||
curve = [shannon_entropy(max(scores[time], 0.0) for time in times if time not in eliminated)]
|
||||
coverage_curve = []
|
||||
width_curve = []
|
||||
for probe in asked:
|
||||
answer = optimal_answer(probe, true_time)
|
||||
if answer is None:
|
||||
curve.append(curve[-1] if curve else 0.0)
|
||||
continue
|
||||
scores, conflicts, eliminated = apply_answer(
|
||||
scores, conflicts, eliminated, probe, answer, times,
|
||||
)
|
||||
remaining, _mode, true_alive = remaining_after_six(
|
||||
all_times=times,
|
||||
scores=scores,
|
||||
eliminated=eliminated,
|
||||
clusters=clusters,
|
||||
true_time=true_time,
|
||||
)
|
||||
active_scores = [max(scores.get(time, 0.0), 0.0) for time in remaining]
|
||||
curve.append(shannon_entropy(active_scores))
|
||||
coverage_curve.append(true_alive)
|
||||
width_curve.append(
|
||||
None if not remaining else (max(len(remaining), 0)),
|
||||
)
|
||||
remaining, remaining_mode, true_alive = remaining_after_six(
|
||||
all_times=times,
|
||||
scores=scores,
|
||||
eliminated=eliminated,
|
||||
clusters=clusters,
|
||||
true_time=true_time,
|
||||
)
|
||||
return {
|
||||
"entropy": curve,
|
||||
"asked": len(asked),
|
||||
"probes": len(initial),
|
||||
"remaining_count": len(remaining),
|
||||
"remaining_mode": remaining_mode,
|
||||
"true_alive": true_alive,
|
||||
"replay_top1": top1_hit(scores, remaining, true_time, clusters),
|
||||
"coverage_after_answers": coverage_curve,
|
||||
}
|
||||
|
||||
|
||||
def score_variant(
|
||||
*,
|
||||
request: dict[str, Any],
|
||||
bundle: dict[str, Any],
|
||||
variant: Variant,
|
||||
true_time: str,
|
||||
) -> dict[str, Any]:
|
||||
matrix = apply_variant(bundle, variant)
|
||||
built = built_payload(bundle, matrix)
|
||||
rows = score_from_matrix(request, built)
|
||||
layers = dynamic_signature_layers(bundle["domains"]) if variant.r5 else SIGNATURE_LAYERS
|
||||
clusters = cluster_contexts(bundle["static_contexts"], layers)
|
||||
public = public_rows(rows, bundle["static_contexts"], layers)
|
||||
prior = {str(row["time"])[:5]: float(row["score"]) for row in rows}
|
||||
engine = metrics_from_public(public, true_time, bundle["times"])
|
||||
replay = entropy_curve(
|
||||
request=request,
|
||||
built=built,
|
||||
prior=prior,
|
||||
true_time=true_time,
|
||||
clusters=clusters,
|
||||
)
|
||||
true_cluster = cluster_of(true_time, clusters)
|
||||
return {
|
||||
"engine": engine,
|
||||
"replay": replay,
|
||||
"true_cluster_key": None if true_cluster is None else true_cluster.get("signature_key"),
|
||||
"true_cluster_size": 0 if true_cluster is None else len(true_cluster.get("times") or []),
|
||||
"score_range": round(
|
||||
max(prior.values()) - min(prior.values()),
|
||||
4,
|
||||
) if prior else 0.0,
|
||||
}
|
||||
|
||||
|
||||
def kp_window_changes(bundle: dict[str, Any]) -> dict[str, int]:
|
||||
changes = {}
|
||||
for domain in sorted(set(bundle["domains"])):
|
||||
changes[domain] = kp_changes_in_window(bundle["static_contexts"], domain)
|
||||
changes["any"] = max(changes.values()) if changes else 0
|
||||
return changes
|
||||
|
||||
|
||||
def summarize(rows: Sequence[dict[str, Any]], *, r3_only: bool = False) -> dict[str, Any]:
|
||||
usable = [row for row in rows if (not r3_only or row.get("kp_eligible"))]
|
||||
if not usable:
|
||||
return {
|
||||
"n": 0,
|
||||
"top1": None,
|
||||
"coverage": None,
|
||||
"width_median": None,
|
||||
"tie": None,
|
||||
"entropy0": None,
|
||||
"entropy6": None,
|
||||
"squeezed": None,
|
||||
"too_narrow": None,
|
||||
"replay_top1": None,
|
||||
}
|
||||
widths = [row["engine"]["width"] for row in usable if row["engine"]["width"] is not None]
|
||||
entropy0 = [row["engine"]["entropy"] for row in usable]
|
||||
entropy6 = [
|
||||
(row["replay"]["entropy"][-1] if row["replay"]["entropy"] else None)
|
||||
for row in usable
|
||||
]
|
||||
entropy6 = [item for item in entropy6 if item is not None]
|
||||
return {
|
||||
"n": len(usable),
|
||||
"top1": round(sum(1 for row in usable if row["engine"]["top1_hit"]) / len(usable), 4),
|
||||
"coverage": round(sum(1 for row in usable if row["engine"]["coverage"]) / len(usable), 4),
|
||||
"width_median": statistics.median(widths) if widths else None,
|
||||
"tie": round(sum(1 for row in usable if row["engine"]["tie"]) / len(usable), 4),
|
||||
"entropy0": round(sum(entropy0) / len(entropy0), 4) if entropy0 else None,
|
||||
"entropy6": round(sum(entropy6) / len(entropy6), 4) if entropy6 else None,
|
||||
"squeezed": sum(1 for row in usable if row["engine"]["truth_squeezed"]),
|
||||
"too_narrow": sum(1 for row in usable if row["engine"]["too_narrow"]),
|
||||
"replay_top1": round(sum(1 for row in usable if row["replay"]["replay_top1"]) / len(usable), 4),
|
||||
}
|
||||
|
||||
|
||||
def verdict(baseline: dict[str, Any], candidate: dict[str, Any]) -> str:
|
||||
if baseline["n"] == 0 or candidate["n"] == 0:
|
||||
return "uncertain"
|
||||
if any(candidate[key] is None or baseline[key] is None for key in ("top1", "coverage", "tie", "width_median")):
|
||||
return "uncertain"
|
||||
hit_ok = candidate["top1"] + 1e-9 >= baseline["top1"]
|
||||
cover_ok = candidate["coverage"] + 1e-9 >= baseline["coverage"]
|
||||
tie_down = candidate["tie"] <= baseline["tie"] + 1e-9
|
||||
width_down = candidate["width_median"] <= baseline["width_median"] + 1e-9
|
||||
squeezed_ok = candidate["squeezed"] <= baseline["squeezed"]
|
||||
if hit_ok and cover_ok and tie_down and width_down and squeezed_ok:
|
||||
if (
|
||||
candidate["top1"] > baseline["top1"] + 1e-9
|
||||
or candidate["tie"] < baseline["tie"] - 1e-9
|
||||
or candidate["width_median"] < baseline["width_median"] - 1e-9
|
||||
):
|
||||
return "benefit"
|
||||
return "no_benefit"
|
||||
if not hit_ok or not cover_ok or candidate["squeezed"] > baseline["squeezed"]:
|
||||
return "no_benefit"
|
||||
return "uncertain"
|
||||
|
||||
|
||||
def md_table(headers: Sequence[str], rows: Sequence[Sequence[Any]]) -> str:
|
||||
def cell(value: Any) -> str:
|
||||
if value is None:
|
||||
return "—"
|
||||
if isinstance(value, float):
|
||||
return f"{value:.3f}".rstrip("0").rstrip(".")
|
||||
return str(value)
|
||||
|
||||
line = "| " + " | ".join(headers) + " |"
|
||||
sep = "| " + " | ".join("---" if index == 0 else "---:" for index in range(len(headers))) + " |"
|
||||
body = ["| " + " | ".join(cell(item) for item in row) + " |" for row in rows]
|
||||
return "\n".join([line, sep, *body])
|
||||
|
||||
|
||||
def write_report(payload: dict[str, Any]) -> None:
|
||||
lines = [
|
||||
"# 分钟分辨率打分尺度测量（2026-09-14）",
|
||||
"",
|
||||
f"- 口径：ayanamsa `{payload['ayanamsa']}`，node mode `{payload['node_mode']}`。不得与上游 true-node 数字直接对比。",
|
||||
f"- 数据：`{payload['holdout']}`，{payload['case_count']} 例公开 Rodden-AA，每例 ≥7 件事、≥4 个领域。",
|
||||
f"- 半径：{', '.join(f'±{item}' for item in payload['radii'])}，步长 {MINUTE_STEP} 分钟。",
|
||||
"- 性质：离线测量。生产 `active_rectification_event_engine.py` / `scoring_service.py` / `candidate_contrast.py` 默认值未改。",
|
||||
"- D1：KP 宫头子主计分方向已拍板；本文件只报告权重收益，不改线上默认。",
|
||||
"",
|
||||
"## 结论",
|
||||
"",
|
||||
]
|
||||
overall = payload["verdicts"]
|
||||
lines.append("| 改法 | 判定 | 说明 |")
|
||||
lines.append("| --- | --- | --- |")
|
||||
for name, row in overall.items():
|
||||
lines.append(f"| {name} | **{row['verdict']}** | {row['note']} |")
|
||||
lines.extend(["", "## 基线成绩单", ""])
|
||||
base_rows = []
|
||||
for radius, summary in payload["baseline"].items():
|
||||
base_rows.append([
|
||||
f"±{radius}",
|
||||
summary["n"],
|
||||
summary["top1"],
|
||||
summary["coverage"],
|
||||
summary["width_median"],
|
||||
summary["tie"],
|
||||
summary["entropy0"],
|
||||
summary["squeezed"],
|
||||
])
|
||||
lines.append(md_table(
|
||||
["半径", "N", "真分钟在头名簇", "真分钟在交付区间", "区间宽度中位", "并列率", "引擎熵", "真值被挤出"],
|
||||
base_rows,
|
||||
))
|
||||
lines.extend(["", "## 分半径指标", ""])
|
||||
for radius, variants in payload["by_radius"].items():
|
||||
lines.append(f"### ±{radius} 分钟")
|
||||
lines.append("")
|
||||
table = []
|
||||
for name, summary in variants.items():
|
||||
table.append([
|
||||
name,
|
||||
summary["top1"],
|
||||
summary["coverage"],
|
||||
summary["width_median"],
|
||||
summary["tie"],
|
||||
summary["entropy0"],
|
||||
summary["entropy6"],
|
||||
summary["squeezed"],
|
||||
summary["too_narrow"],
|
||||
])
|
||||
lines.append(md_table(
|
||||
["方案", "头名簇命中", "区间覆盖", "宽度中位", "并列率", "熵0", "熵6", "挤出", "过窄"],
|
||||
table,
|
||||
))
|
||||
lines.append("")
|
||||
lines.extend([
|
||||
"## R3 KP 窗内变化",
|
||||
"",
|
||||
"只统计 ±10 窗内目标宫宫头子主至少变化一次的例子。",
|
||||
"",
|
||||
])
|
||||
kp = payload["kp"]
|
||||
lines.append(md_table(
|
||||
["例子", "窗内变化次数", "计入 R3"],
|
||||
[[row["case_id"], row["changes"], "yes" if row["eligible"] else "no"] for row in kp],
|
||||
))
|
||||
lines.extend([
|
||||
"",
|
||||
"## 方法",
|
||||
"",
|
||||
"1. 每个例子先算生产静态盘（含 KP 观察），再按事件抽样日打生产分。",
|
||||
"2. R1 按 `*_domain_varga` 规则把 `2*len` 换成 `len` / `sqrt(len)` / 固定 2。",
|
||||
"3. R2 在事件级减去当前窗口最小值。",
|
||||
"4. R3 把目标宫 KP 子主/次子主与当时 Vim MD/AD/PD 比对，权重 0.5 / 1.0 / 2.0。",
|
||||
"5. R4 对年精度 12 个月样本取 max 或 log-sum-exp（T=1）。",
|
||||
"6. R5 按本例有证据的领域动态取签名层，去掉同日恒定的 `md`。",
|
||||
"7. 交付区间 = 公开簇成员分钟的并集。并列 = 头名簇分数并列。",
|
||||
"8. 熵曲线用生产探针 + 真值方向最优答，最多 6 题。",
|
||||
"",
|
||||
"配对组合用 R1=`len`、R3=1.0、R4=`lse`。",
|
||||
"",
|
||||
])
|
||||
REPORT_MD.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
REPORT_JSON.write_text(json.dumps(payload, ensure_ascii=True, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def run(args: argparse.Namespace) -> dict[str, Any]:
|
||||
holdout = json.loads(HOLDOUT.read_text(encoding="utf-8"))
|
||||
cases = list(holdout["cases"])
|
||||
if args.limit:
|
||||
cases = cases[: args.limit]
|
||||
radii = tuple(int(item) for item in args.radii)
|
||||
variants = all_variants()
|
||||
if args.quick:
|
||||
variants = [item for item in variants if item.name in {"baseline", "R1@len", "R2", "R3@1.0", "R4@lse", "R5"}]
|
||||
results: dict[str, dict[str, list[dict[str, Any]]]] = {
|
||||
str(radius): {variant.name: [] for variant in variants} for radius in radii
|
||||
}
|
||||
kp_rows = []
|
||||
errors = []
|
||||
for case in cases:
|
||||
case_id = case["case_id"]
|
||||
true_time = str(case["birth"]["time"])[:5]
|
||||
print(f"case {case_id}", flush=True)
|
||||
try:
|
||||
for radius in radii:
|
||||
request = scoring_request_for(case, radius)
|
||||
contexts = compute_candidate_static_contexts(request)
|
||||
bundle = collect_samples(request, contexts)
|
||||
changes = kp_window_changes(bundle)
|
||||
eligible = changes["any"] > 0
|
||||
if radius == 10:
|
||||
kp_rows.append({
|
||||
"case_id": case_id,
|
||||
"changes": changes["any"],
|
||||
"by_domain": changes,
|
||||
"eligible": eligible,
|
||||
})
|
||||
print(f" radius ±{radius} candidates={len(bundle['times'])}", flush=True)
|
||||
for variant in variants:
|
||||
scored = score_variant(
|
||||
request=request,
|
||||
bundle=bundle,
|
||||
variant=variant,
|
||||
true_time=true_time,
|
||||
)
|
||||
scored["case_id"] = case_id
|
||||
scored["kp_eligible"] = eligible
|
||||
results[str(radius)][variant.name].append(scored)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
errors.append({"case_id": case_id, "error": f"{type(exc).__name__}: {exc}", "trace": traceback.format_exc()})
|
||||
print(f" FAIL {case_id}: {exc}", flush=True)
|
||||
by_radius = {}
|
||||
baseline = {}
|
||||
verdicts: dict[str, dict[str, str]] = {}
|
||||
for radius in radii:
|
||||
variant_summaries = {}
|
||||
for variant in variants:
|
||||
rows = results[str(radius)][variant.name]
|
||||
variant_summaries[variant.name] = summarize(rows, r3_only=variant.uses_r3)
|
||||
by_radius[str(radius)] = variant_summaries
|
||||
baseline[str(radius)] = variant_summaries["baseline"]
|
||||
names = [variant.name for variant in variants if variant.name != "baseline"]
|
||||
for name in names:
|
||||
notes = []
|
||||
labels = []
|
||||
for radius in radii:
|
||||
base = by_radius[str(radius)]["baseline"]
|
||||
cand = by_radius[str(radius)][name]
|
||||
label = verdict(base, cand)
|
||||
labels.append(label)
|
||||
notes.append(f"±{radius}:{label}")
|
||||
if all(item == "benefit" for item in labels):
|
||||
final = "benefit"
|
||||
elif all(item == "no_benefit" for item in labels):
|
||||
final = "no_benefit"
|
||||
else:
|
||||
final = "uncertain"
|
||||
verdicts[name] = {"verdict": final, "note": "；".join(notes)}
|
||||
payload = {
|
||||
"generated_at": TODAY.isoformat(),
|
||||
"ayanamsa": AYANAMSA,
|
||||
"node_mode": NODE_MODE,
|
||||
"holdout": str(HOLDOUT.relative_to(ROOT)).replace("\\", "/"),
|
||||
"case_count": len(cases),
|
||||
"radii": list(radii),
|
||||
"minute_step": MINUTE_STEP,
|
||||
"baseline": baseline,
|
||||
"by_radius": by_radius,
|
||||
"verdicts": verdicts,
|
||||
"kp": kp_rows,
|
||||
"errors": errors,
|
||||
"results": results,
|
||||
}
|
||||
write_report(payload)
|
||||
return payload
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--limit", type=int, default=0)
|
||||
parser.add_argument("--radii", nargs="+", default=[str(item) for item in RADII])
|
||||
parser.add_argument("--quick", action="store_true")
|
||||
args = parser.parse_args()
|
||||
payload = run(args)
|
||||
print(f"wrote {REPORT_MD} cases={payload['case_count']} errors={len(payload['errors'])}")
|
||||
return 0 if not payload["errors"] else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,97 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from scripts.research.holdout_v4_build import MIN_DOMAINS, MIN_EVENTS, build
|
||||
from scripts.research.minute_resolution_lib import (
|
||||
aggregate_samples,
|
||||
dynamic_signature_layers,
|
||||
rescale_varga_points,
|
||||
shannon_entropy,
|
||||
subtract_event_floor,
|
||||
varga_divisor,
|
||||
)
|
||||
from scripts.research.minute_resolution_sweep import all_variants, verdict
|
||||
|
||||
|
||||
def test_v4_cases_meet_event_and_domain_floors() -> None:
|
||||
payload = build()
|
||||
assert payload["ayanamsa"] == "raman"
|
||||
assert payload["node_mode"] == "mean"
|
||||
assert len(payload["cases"]) >= 8
|
||||
for case in payload["cases"]:
|
||||
domains = {item["domain"] for item in case["events"]}
|
||||
assert len(case["events"]) >= MIN_EVENTS, case["case_id"]
|
||||
assert len(domains) >= MIN_DOMAINS, case["case_id"]
|
||||
assert case["birth"]["source"]["rodden_rating"] == "AA"
|
||||
assert str(case["birth"]["time"])[:5] == case["true_minute"]
|
||||
|
||||
|
||||
def test_varga_divisor_modes() -> None:
|
||||
assert varga_divisor(1, "2len") == 2.0
|
||||
assert varga_divisor(3, "2len") == 6.0
|
||||
assert varga_divisor(3, "len") == 3.0
|
||||
assert abs(varga_divisor(4, "sqrt") - 2.0) < 1e-9
|
||||
assert varga_divisor(3, "fixed2") == 2.0
|
||||
|
||||
|
||||
def test_year_aggregation_keeps_peak() -> None:
|
||||
samples = [1.0, 1.0, 12.0]
|
||||
assert aggregate_samples(samples, "mean") == 4.6667
|
||||
assert aggregate_samples(samples, "max") == 12.0
|
||||
lse = aggregate_samples(samples, "lse", 1.0)
|
||||
assert lse > aggregate_samples(samples, "mean")
|
||||
assert lse < 12.0
|
||||
|
||||
|
||||
def test_subtract_event_floor_zeroes_the_min() -> None:
|
||||
out = subtract_event_floor({"04:50": 11.0, "04:51": 13.5, "04:52": 11.0})
|
||||
assert out["04:50"] == 0.0
|
||||
assert out["04:51"] == 2.5
|
||||
|
||||
|
||||
def test_entropy_is_zero_for_a_spike() -> None:
|
||||
assert shannon_entropy([0, 0, 5]) == 0.0
|
||||
assert shannon_entropy([1, 1]) > 0.9
|
||||
|
||||
|
||||
def test_varga_rescale_doubles_single_chart_hit() -> None:
|
||||
points = rescale_varga_points(
|
||||
1.0,
|
||||
["vim_md_domain_varga", "event_kind:career_change"],
|
||||
"career",
|
||||
"career_change",
|
||||
"day",
|
||||
"len",
|
||||
)
|
||||
assert points > 1.0
|
||||
|
||||
|
||||
def test_dynamic_layers_drop_md_and_follow_domains() -> None:
|
||||
layers = dynamic_signature_layers(["career", "health_pressure"])
|
||||
assert layers[0] == "d1"
|
||||
assert "d10" in layers
|
||||
assert "d30" in layers
|
||||
assert "md" not in layers
|
||||
|
||||
|
||||
def test_variant_grid_covers_pairs() -> None:
|
||||
names = {item.name for item in all_variants()}
|
||||
assert "baseline" in names
|
||||
assert "R1@len" in names
|
||||
assert "R3@1.0" in names
|
||||
assert "R1+R3" in names
|
||||
pair = next(item for item in all_variants() if item.name == "R1+R3")
|
||||
assert pair.r1 == "len"
|
||||
assert pair.r3 == 1.0
|
||||
|
||||
|
||||
def test_verdict_requires_hit_and_coverage() -> None:
|
||||
baseline = {
|
||||
"n": 10, "top1": 0.2, "coverage": 0.9, "width_median": 21, "tie": 0.8,
|
||||
"squeezed": 1, "entropy0": 3.0, "entropy6": 2.0, "too_narrow": 0, "replay_top1": 0.5,
|
||||
}
|
||||
better = {
|
||||
**baseline, "top1": 0.4, "tie": 0.4, "width_median": 12, "squeezed": 0,
|
||||
}
|
||||
worse = {**baseline, "top1": 0.1, "coverage": 0.7, "squeezed": 3}
|
||||
assert verdict(baseline, better) == "benefit"
|
||||
assert verdict(baseline, worse) == "no_benefit"
|
||||
Reference in New Issue
Block a user