docs(research): measure precision-adaptive probe gates; no variant clears the bar
20 public AA cases, raman/mean. Lowering MIN_BOUNDARY_DAYS narrows ±10 from 15 to 11 minutes but drops top-1 0.80→0.75; wider radii get wider ranges. Varga sensitivity weights (V1/V2) match production; D60 (V3) hurts ±10. Day-precision events offset ±7 never squeeze the true minute out. Production 45/30 gate and equal varga weights stay unchanged.
This commit is contained in:
jesse-ux
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,277 @@
|
||||
# 出题闸门按证据精度分档测量（2026-09-14）
|
||||
|
||||
- 口径：ayanamsa `raman`，node mode `mean`。不得与上游 true-node 数字直接对比。
|
||||
- 数据：`references/real_case_calibration/minute_rectification_holdout_v4.json`，20 例公开 Rodden-AA。精度 日 59 / 月 0 / 年 84。
|
||||
- 半径：±10, ±30, ±60，步长 2 分钟。
|
||||
- 性质：离线测量。生产 `event_probes.py` 的 `MIN_BOUNDARY_DAYS=45` / `REFRESH_MIN_BOUNDARY_DAYS=30` 未改；计分引擎等权分盘未改。
|
||||
- 线上对照：`unionStillValidRange` = 未淘汰且落后头名不足 8 分的簇覆盖并集。
|
||||
|
||||
## 结论
|
||||
|
||||
**无收益，不立实现单。** 线上 `MIN_BOUNDARY_DAYS=45` / 刷新 30 与等权分盘保持不动。真机卡住的 ±10 窗上，放宽闸门能把宽度从 15 收到 11，但头名从 0.80 掉到 0.75；宽窗上宽度反而变大。V1/V2 与基线相同，V3（D60）在 ±10 把头名降到 0.70。唯一看起来像过门的格子是 G1+V3 在 ±10（命中仍 0.80、宽度 15→11），宽窗上不成立，也没有对该组合做 ±7 记错，不够立实现单。
|
||||
|
||||
判定口径：命中不降 **且** 宽度下降才算有收益；只多出题而宽度变宽不算。±7 天记错挤出真值的档位一律不推荐。本网格覆盖率始终 20/20，M2 挤出全是 0。
|
||||
|
||||
| 方案 | 判定 | 说明 |
|
||||
| --- | ---: | ---: |
|
||||
| G1 | no_benefit | ±10:no_benefit squeezed7=0；±30:no_benefit squeezed7=0；±60:no_benefit squeezed7=0 |
|
||||
| G2 | no_benefit | ±10:no_benefit squeezed7=0；±30:no_benefit squeezed7=0；±60:no_benefit squeezed7=0 |
|
||||
| G3 | no_benefit | ±10:no_benefit squeezed7=0；±30:no_benefit squeezed7=0；±60:no_benefit squeezed7=0 |
|
||||
| G4 | no_benefit | ±10:no_benefit squeezed7=0；±30:no_benefit squeezed7=0；±60:no_benefit squeezed7=0 |
|
||||
| V1 | no_benefit | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| V2 | no_benefit | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
| V3 | no_benefit | ±10:no_benefit；±30:no_benefit；±60:no_benefit |
|
||||
|
||||
过运与三大外部引擎未参与本单（纯出题闸门 / 分盘配权，不解释运势）。
|
||||
|
||||
## M0 · 精度处理三组（生产闸门）
|
||||
|
||||
同一批例子、生产 45/30 闸。A 原样，B 日→月，C 全部→年。用来隔离精度本身，而不是闸门。
|
||||
|
||||
### ±10
|
||||
|
||||
| 组 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 | 首轮出题均 | 六题后再出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| A | 0.8 | 1 | 15 | 0 | 0 | 5.1 | 3.25 |
|
||||
| B | 0.8 | 1 | 15 | 0 | 0 | 5.1 | 3.35 |
|
||||
| C | 0.75 | 1 | 15 | 0 | 0 | 5.1 | 3.3 |
|
||||
|
||||
### ±30
|
||||
|
||||
| 组 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 | 首轮出题均 | 六题后再出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| A | 0.55 | 1 | 33 | 0 | 0 | 7.1 | 5.45 |
|
||||
| B | 0.55 | 1 | 35 | 0 | 0 | 7.1 | 5.2 |
|
||||
| C | 0.6 | 1 | 35 | 0 | 0 | 7.1 | 5.2 |
|
||||
|
||||
### ±60
|
||||
|
||||
| 组 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 | 首轮出题均 | 六题后再出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| A | 0.4 | 1 | 56 | 0 | 0 | 7.2 | 5.75 |
|
||||
| B | 0.4 | 1 | 58 | 0 | 0 | 7.2 | 6.05 |
|
||||
| C | 0.45 | 1 | 61 | 0 | 0 | 7.2 | 6.1 |
|
||||
|
||||
对 BUG-689 邀请文案：组 A 与把日精度降成月的组 B 在 ±10 上命中/宽度完全一样（0.80 / 15）。全部降成年精度的组 C 命中只掉到 0.75。v4 没有原生月精度事件，**不能**把「记得到天比记得到月有用得多」写成已证实，更不能承诺能定到分钟。
|
||||
|
||||
## M1 · 闸门分档（组 A）
|
||||
|
||||
### ±10
|
||||
|
||||
| 档 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 | 首轮出题均 | 六题后再出 | 有分辨力 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| G0 | 0.8 | 1 | 15 | 0 | 0 | 5.1 | 3.25 | 3.25 |
|
||||
| G1 | 0.75 | 1 | 11 | 0 | 0 | 5.9 | 3.3 | 3.3 |
|
||||
| G2 | 0.75 | 1 | 11 | 0 | 0 | 5.9 | 2.95 | 2.95 |
|
||||
| G3 | 0.75 | 1 | 11 | 0 | 0 | 5.9 | 2.95 | 2.95 |
|
||||
| G4 | 0.75 | 1 | 11 | 0 | 0 | 5.9 | 2.95 | 2.95 |
|
||||
|
||||
### ±30
|
||||
|
||||
| 档 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 | 首轮出题均 | 六题后再出 | 有分辨力 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| G0 | 0.55 | 1 | 33 | 0 | 0 | 7.1 | 5.45 | 5.45 |
|
||||
| G1 | 0.55 | 1 | 35 | 0 | 0 | 7.05 | 5.4 | 5.4 |
|
||||
| G2 | 0.55 | 1 | 37 | 0 | 0 | 7.05 | 5.5 | 5.5 |
|
||||
| G3 | 0.55 | 1 | 37 | 0 | 0 | 7.05 | 5.5 | 5.5 |
|
||||
| G4 | 0.55 | 1 | 37 | 0 | 0 | 7.05 | 5.6 | 5.6 |
|
||||
|
||||
### ±60
|
||||
|
||||
| 档 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 | 首轮出题均 | 六题后再出 | 有分辨力 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| G0 | 0.4 | 1 | 56 | 0 | 0 | 7.2 | 5.75 | 5.4 |
|
||||
| G1 | 0.35 | 1 | 58 | 0 | 0 | 7.2 | 6.1 | 6.1 |
|
||||
| G2 | 0.4 | 1 | 64 | 0 | 0 | 7.2 | 6.25 | 6.25 |
|
||||
| G3 | 0.4 | 1 | 64 | 0 | 0 | 7.2 | 6.2 | 6.2 |
|
||||
| G4 | 0.4 | 1 | 64 | 0 | 0 | 7.2 | 6.25 | 6.25 |
|
||||
|
||||
## M1b · 分盘按分钟敏感度配权
|
||||
|
||||
V0 生产等权；V1 `min(window/varga_minutes, 1)` 再除以 `2n`；V2 只计窗内至少变一次的盘；V3 = V2 + D60（仅 ±10）。
|
||||
未出现在上游表里的 D2/D3/D5/D7/D11 用 `120/n` 分钟。按盘贡献表本轮是空的：V0 采集误走了生产计分、没有打上分盘标签；但 V1/V2 三档半径都与 V0 数字相同，V3 只在 ±10 把命中从 0.80 降到 0.70，已经够判断这三档不能上线。
|
||||
|
||||
### ±10
|
||||
|
||||
| 改法 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| V0 | 0.8 | 1 | 15 | 0 | 0 |
|
||||
| V1 | 0.8 | 1 | 15 | 0 | 0 |
|
||||
| V2 | 0.8 | 1 | 15 | 0 | 0 |
|
||||
| V3 | 0.7 | 1 | 15 | 0 | 0 |
|
||||
|
||||
### ±30
|
||||
|
||||
| 改法 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| V0 | 0.55 | 1 | 33 | 0 | 0 |
|
||||
| V1 | 0.55 | 1 | 33 | 0 | 0 |
|
||||
| V2 | 0.55 | 1 | 33 | 0 | 0 |
|
||||
| V3 | 0.55 | 1 | 33 | 0 | 0 |
|
||||
|
||||
### ±60
|
||||
|
||||
| 改法 | 头名命中 | 区间覆盖 | 宽度中位 | 并列率 | 挤出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| V0 | 0.4 | 1 | 56 | 0 | 0 |
|
||||
| V1 | 0.4 | 1 | 56 | 0 | 0 |
|
||||
| V2 | 0.4 | 1 | 56 | 0 | 0 |
|
||||
| V3 | 0.4 | 1 | 56 | 0 | 0 |
|
||||
|
||||
### 闸门 × 分盘组合（组 A）
|
||||
|
||||
±10
|
||||
|
||||
| 组合 | 头名命中 | 区间覆盖 | 宽度中位 | 挤出 | 六题后再出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| G1+V1 | 0.75 | 1 | 11 | 0 | 3.3 |
|
||||
| G2+V1 | 0.75 | 1 | 11 | 0 | 2.95 |
|
||||
| G3+V1 | 0.75 | 1 | 11 | 0 | 2.95 |
|
||||
| G4+V1 | 0.75 | 1 | 11 | 0 | 2.95 |
|
||||
| G1+V2 | 0.75 | 1 | 11 | 0 | 3.3 |
|
||||
| G2+V2 | 0.75 | 1 | 11 | 0 | 2.95 |
|
||||
| G3+V2 | 0.75 | 1 | 11 | 0 | 2.95 |
|
||||
| G4+V2 | 0.75 | 1 | 11 | 0 | 2.95 |
|
||||
| G1+V3 | 0.8 | 1 | 11 | 0 | 3.2 |
|
||||
| G2+V3 | 0.8 | 1 | 11 | 0 | 2.85 |
|
||||
| G3+V3 | 0.8 | 1 | 11 | 0 | 2.85 |
|
||||
| G4+V3 | 0.8 | 1 | 11 | 0 | 2.85 |
|
||||
|
||||
±30
|
||||
|
||||
| 组合 | 头名命中 | 区间覆盖 | 宽度中位 | 挤出 | 六题后再出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| G1+V1 | 0.55 | 1 | 35 | 0 | 5.4 |
|
||||
| G2+V1 | 0.55 | 1 | 37 | 0 | 5.5 |
|
||||
| G3+V1 | 0.55 | 1 | 37 | 0 | 5.5 |
|
||||
| G4+V1 | 0.55 | 1 | 37 | 0 | 5.6 |
|
||||
| G1+V2 | 0.55 | 1 | 35 | 0 | 5.4 |
|
||||
| G2+V2 | 0.55 | 1 | 37 | 0 | 5.5 |
|
||||
| G3+V2 | 0.55 | 1 | 37 | 0 | 5.5 |
|
||||
| G4+V2 | 0.55 | 1 | 37 | 0 | 5.6 |
|
||||
| G1+V3 | 0.55 | 1 | 35 | 0 | 5.4 |
|
||||
| G2+V3 | 0.55 | 1 | 37 | 0 | 5.5 |
|
||||
| G3+V3 | 0.55 | 1 | 37 | 0 | 5.5 |
|
||||
| G4+V3 | 0.55 | 1 | 37 | 0 | 5.6 |
|
||||
|
||||
±60
|
||||
|
||||
| 组合 | 头名命中 | 区间覆盖 | 宽度中位 | 挤出 | 六题后再出 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| G1+V1 | 0.35 | 1 | 58 | 0 | 6.1 |
|
||||
| G2+V1 | 0.4 | 1 | 64 | 0 | 6.25 |
|
||||
| G3+V1 | 0.4 | 1 | 64 | 0 | 6.2 |
|
||||
| G4+V1 | 0.4 | 1 | 64 | 0 | 6.25 |
|
||||
| G1+V2 | 0.35 | 1 | 58 | 0 | 6.1 |
|
||||
| G2+V2 | 0.4 | 1 | 64 | 0 | 6.25 |
|
||||
| G3+V2 | 0.4 | 1 | 64 | 0 | 6.2 |
|
||||
| G4+V2 | 0.4 | 1 | 64 | 0 | 6.25 |
|
||||
| G1+V3 | 0.35 | 1 | 58 | 0 | 6.1 |
|
||||
| G2+V3 | 0.4 | 1 | 64 | 0 | 6.25 |
|
||||
| G3+V3 | 0.4 | 1 | 64 | 0 | 6.2 |
|
||||
| G4+V3 | 0.4 | 1 | 64 | 0 | 6.25 |
|
||||
|
||||
## M2 · 答错容忍度（日精度事件日期偏移）
|
||||
|
||||
组 A 的日精度事件按例、按事件独立随机偏移（种子 20260914，排除 0）。±7 天出现真值挤出的档位不得推荐。
|
||||
|
||||
### ±3 天
|
||||
|
||||
| 格子 | 头名命中 | 区间覆盖 | 挤出 | 宽度中位 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| ±10 G0 | 0.8 | 1 | 0 | 14 |
|
||||
| ±10 G1 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G2 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G3 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G4 | 0.75 | 1 | 0 | 11 |
|
||||
| ±30 G0 | 0.55 | 1 | 0 | 34 |
|
||||
| ±30 G1 | 0.55 | 1 | 0 | 35 |
|
||||
| ±30 G2 | 0.55 | 1 | 0 | 35 |
|
||||
| ±30 G3 | 0.55 | 1 | 0 | 35 |
|
||||
| ±30 G4 | 0.55 | 1 | 0 | 35 |
|
||||
| ±60 G0 | 0.4 | 1 | 0 | 56 |
|
||||
| ±60 G1 | 0.4 | 1 | 0 | 55 |
|
||||
| ±60 G2 | 0.45 | 1 | 0 | 56 |
|
||||
| ±60 G3 | 0.45 | 1 | 0 | 56 |
|
||||
| ±60 G4 | 0.45 | 1 | 0 | 56 |
|
||||
|
||||
### ±7 天
|
||||
|
||||
| 格子 | 头名命中 | 区间覆盖 | 挤出 | 宽度中位 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| ±10 G0 | 0.75 | 1 | 0 | 14 |
|
||||
| ±10 G1 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G2 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G3 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G4 | 0.75 | 1 | 0 | 11 |
|
||||
| ±30 G0 | 0.6 | 1 | 0 | 35 |
|
||||
| ±30 G1 | 0.5 | 1 | 0 | 35 |
|
||||
| ±30 G2 | 0.5 | 1 | 0 | 35 |
|
||||
| ±30 G3 | 0.5 | 1 | 0 | 35 |
|
||||
| ±30 G4 | 0.5 | 1 | 0 | 35 |
|
||||
| ±60 G0 | 0.4 | 1 | 0 | 83 |
|
||||
| ±60 G1 | 0.35 | 1 | 0 | 61 |
|
||||
| ±60 G2 | 0.45 | 1 | 0 | 64 |
|
||||
| ±60 G3 | 0.4 | 1 | 0 | 59 |
|
||||
| ±60 G4 | 0.4 | 1 | 0 | 59 |
|
||||
|
||||
### ±14 天
|
||||
|
||||
| 格子 | 头名命中 | 区间覆盖 | 挤出 | 宽度中位 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| ±10 G0 | 0.75 | 1 | 0 | 15 |
|
||||
| ±10 G1 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G2 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G3 | 0.75 | 1 | 0 | 11 |
|
||||
| ±10 G4 | 0.75 | 1 | 0 | 11 |
|
||||
| ±30 G0 | 0.65 | 1 | 0 | 35 |
|
||||
| ±30 G1 | 0.5 | 1 | 0 | 37 |
|
||||
| ±30 G2 | 0.5 | 1 | 0 | 37 |
|
||||
| ±30 G3 | 0.5 | 1 | 0 | 37 |
|
||||
| ±30 G4 | 0.5 | 1 | 0 | 37 |
|
||||
| ±60 G0 | 0.45 | 1 | 0 | 58 |
|
||||
| ±60 G1 | 0.4 | 1 | 0 | 76 |
|
||||
| ±60 G2 | 0.5 | 1 | 0 | 64 |
|
||||
| ±60 G3 | 0.5 | 1 | 0 | 64 |
|
||||
| ±60 G4 | 0.5 | 1 | 0 | 64 |
|
||||
|
||||
V3 在 ±7 天偏移下：
|
||||
|
||||
| 半径 | 头名命中 | 区间覆盖 | 挤出 |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| ±10 | 0.7 | 1 | 0 |
|
||||
| ±30 | 0.6 | 1 | 0 |
|
||||
| ±60 | 0.4 | 1 | 0 |
|
||||
|
||||
## M3 · 六题之后还能再出几道
|
||||
|
||||
数字见 M1 表的「六题后再出」与「有分辨力」。若新增 ≥8 道而宽度只收 1–2 分钟，判定里记为问答成本高于收益。
|
||||
|
||||
| 半径 | 档 | 再出均 | 有分辨力均 | 宽度中位 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| ±10 | G0 | 3.25 | 3.25 | 15 |
|
||||
| ±10 | G1 | 3.3 | 3.3 | 11 |
|
||||
| ±10 | G2 | 2.95 | 2.95 | 11 |
|
||||
| ±10 | G3 | 2.95 | 2.95 | 11 |
|
||||
| ±10 | G4 | 2.95 | 2.95 | 11 |
|
||||
| ±30 | G0 | 5.45 | 5.45 | 33 |
|
||||
| ±30 | G1 | 5.4 | 5.4 | 35 |
|
||||
| ±30 | G2 | 5.5 | 5.5 | 37 |
|
||||
| ±30 | G3 | 5.5 | 5.5 | 37 |
|
||||
| ±30 | G4 | 5.6 | 5.6 | 37 |
|
||||
| ±60 | G0 | 5.75 | 5.4 | 56 |
|
||||
| ±60 | G1 | 6.1 | 6.1 | 58 |
|
||||
| ±60 | G2 | 6.25 | 6.25 | 64 |
|
||||
| ±60 | G3 | 6.2 | 6.2 | 64 |
|
||||
| ±60 | G4 | 6.25 | 6.25 | 64 |
|
||||
|
||||
## 方法
|
||||
|
||||
1. 每例先算一次静态盘（与事件无关），再按精度处理 / 偏移 / 分盘政策套 `build_event_contribution_matrix`。
|
||||
2. 闸门只通过研究脚本临时改 `MIN_BOUNDARY_DAYS` / `REFRESH_MIN_BOUNDARY_DAYS`，函数返回后立刻恢复。
|
||||
3. 参与该题的证据精度取该例事件包里最细的一档（有日用日，否则月，否则年）。
|
||||
4. 六题回放用真值簇最优 yes/no（上界，不是真人会怎么答）。交付口径与簇宽度研究相同：lead 8 未淘汰并集。
|
||||
5. 刷新出题走生产 `refresh_probes=true`（含已落地的 R3/R4），只换边界天数闸。
|
||||
|
||||
错误 0 例。20 例跑完。
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# 进度 · 出题闸门按证据精度分档研究（2026-09-14）
|
||||
|
||||
## 范围
|
||||
|
||||
- 分支：`codex/rectification-precision-gate-research-20260914`（基于当时 `origin/staging`；任务书含 M1b 的 `b56147e5`）
|
||||
- 任务单：`docs/tasks/TASK-rectification-precision-adaptive-boundary-research-20260914.md`
|
||||
- 结果：`docs/research/precision_gate_2026_09_14.md` + 同名 `.json`
|
||||
- 脚本：`scripts/research/precision_gate_lib.py`、`scripts/research/precision_gate_sweep.py`（离线，不进 API）
|
||||
- 未改 `scripts/rectification/event_probes.py` 线上默认值（仍是 45 / 30），未改计分引擎等权分盘
|
||||
- 本单不占 BUG 号
|
||||
|
||||
## 完成
|
||||
|
||||
- 开工读了结论页、`probe_supply_after_six`、`_boundary_windows` / `_union_boundary_dates`，以及更新后的 M1b。
|
||||
- 离线测量：M0 精度三组、M1 G0–G4、M1b V1/V2/V3（±10 才加 D60）、M2 ±3/±7/±14、M3 六题后再出题。闸门只在脚本里临时 patch。
|
||||
- 全量 20 例 × ±10/±30/±60，错误 0，约 2.5 小时。
|
||||
- 判定函数初版把「多出分辨题」也算收益，宽窗宽度其实变大；已改成任务书口径（命中不降 **且** 宽度下降），用 `--from-json` 重写结论，**没有重跑引擎**。
|
||||
|
||||
## 验收数字
|
||||
|
||||
口径 ayanamsa=`raman`、node mode=`mean`，步长 2 分钟。覆盖率三档都是 20/20，M2 挤出全是 0。
|
||||
|
||||
六题后线上口径（组 A，生产闸 G0）：
|
||||
|
||||
| 半径 | 头名命中 | 宽度中位 | 六题后再出 |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| ±10 | 0.80 | 15 | 3.25 |
|
||||
| ±30 | 0.55 | 33 | 5.45 |
|
||||
| ±60 | 0.40 | 56 | 5.75 |
|
||||
|
||||
放宽闸门（G1–G4）在 ±10：宽度 15→11，头名 0.80→0.75。±30/±60 宽度变宽（33→35–37，56→58–64）。
|
||||
|
||||
M1b：V1/V2 与 V0 相同；V3 在 ±10 命中 0.80→0.70。
|
||||
|
||||
M0：日→月（B）与原样（A）在 ±10 相同；全部降成年（C）命中 0.80→0.75。
|
||||
|
||||
**判定：无收益，不立实现单。**
|
||||
|
||||
- `python -m pytest tests/test_precision_gate_research.py tests/test_rectification_refresh_r3_r4.py`：16 passed
|
||||
- 按盘贡献表空（V0 采集误走生产计分）；不改变 V1=V2=V0、V3 降命中的结论
|
||||
- 本机无项目 `.venv`，用 Anaconda Python 3.11.7 + swisseph。`run_quality_gate.py --profile quick` 若因 `mcp` 包缺口退出，与基线环境相同，不计入
|
||||
|
||||
## 未做
|
||||
|
||||
- 未改线上闸门或分盘权重
|
||||
- 未把研究脚本接到 API
|
||||
- 未立实现单
|
||||
- 未做 PyJHora / VedAstro / jyotishganit 对照（本单不解释运势）
|
||||
- 未 push
|
||||
@@ -190,7 +190,7 @@
|
||||
|
||||
| `TASK-rectification-birth-time-provenance-20260914.md` | `PROGRESS-rectification-birth-time-provenance-20260914.md` | 与上游流程对照发现的缺口（上游 R2）：录入时已问「医院记录 / 家人记得大概 / 只知道时段」并落库 `birth_time_source`，但整条校正链只在放宽窗口时用过一次，决策与文案一律不看它——家人推算的时间和出生证在系统里等同对待，输出还直接称「你的出生时间」。本单把来源标签接进投影并分档措辞（BUG-690） | 待验收 | `codex/rectification-birth-time-provenance-20260914` |
|
||||
|
||||
| `TASK-rectification-precision-adaptive-boundary-research-20260914.md` | `PROGRESS-rectification-precision-gate-research-20260914.md` | **研究单**：出题闸门 `MIN_BOUNDARY_DAYS=45`（刷新 30）不看证据精度，而出生时间每差 1 分钟大运边界只平移约 1.1 天——20 分钟窗内的候选边界全挤在三周内被整批丢掉，这就是「六题后出不来题」的算术原因。量闸门按年/月/日分档（G0–G4）的收益，并必测答错容忍度（±3/±7/±14 天偏移下真值是否被挤出） | 待执行 | `codex/rectification-precision-gate-research-20260914` |
|
||||
| `TASK-rectification-precision-adaptive-boundary-research-20260914.md` | `PROGRESS-rectification-precision-gate-research-20260914.md` | **研究单**：出题闸门 `MIN_BOUNDARY_DAYS=45`（刷新 30）不看证据精度；另加 M1b 按分盘分钟敏感度配权（V1/V2/V3，含 D60）。量 G0–G4 与 V1–V3，并必测 ±3/±7/±14 天记错是否挤出真值 | 待验收 | `codex/rectification-precision-gate-research-20260914` |
|
||||
|
||||
| `TASK-rectification-open-collect-invite-20260914.md` | `PROGRESS-rectification-open-collect-invite-20260914.md` | **P0**：固定七条采集线问完后只说「能问的都问完了」，用户不知道还能补经历、也不知道补了有用；而两轮研究证明补带年月经历是唯一有效手段。产品拍板：交付卡照出 + 卡上给不限领域的补充邀请（先要确切日期，再退年月；举七条线之外的例子），补完必须可见生效（BUG-689） | 待验收 | `codex/rectification-open-collect-invite-20260914` |
|
||||
|
||||
|
||||
@@ -0,0 +1,532 @@
|
||||
"""Pure helpers for the precision-adaptive probe-gate research sweep.
|
||||
|
||||
Does not change production defaults in event_probes.py or the scoring engine.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from copy import deepcopy
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import date, timedelta
|
||||
from random import Random
|
||||
from typing import Any, Iterator, Sequence
|
||||
|
||||
from scripts.active_rectification_event_engine import (
|
||||
AUXILIARY_DOMAINS,
|
||||
AUXILIARY_SCORE_FACTOR,
|
||||
DOMAIN_CONFIG,
|
||||
OBSERVATION_ONLY_LAYERS,
|
||||
_active_narayana,
|
||||
_active_vimshottari,
|
||||
_ashtakavarga_auxiliary,
|
||||
_controlled_transit_rules,
|
||||
_event_datetime,
|
||||
_house_lords,
|
||||
_planet_house,
|
||||
_relative_house,
|
||||
_shadbala_verified_components_auxiliary,
|
||||
_varga_chart,
|
||||
_varga_house,
|
||||
)
|
||||
from scripts.active_rectification_events import precision_weight
|
||||
from scripts.rectification.event_probes import (
|
||||
REFRESH_MIN_BOUNDARY_DAYS,
|
||||
_boundary_windows,
|
||||
)
|
||||
import functional_benefics
|
||||
import varga
|
||||
|
||||
GATES: dict[str, dict[str, int]] = {
|
||||
"G0": {"year": 45, "month": 45, "day": 45},
|
||||
"G1": {"year": 45, "month": 30, "day": 10},
|
||||
"G2": {"year": 45, "month": 30, "day": 7},
|
||||
"G3": {"year": 45, "month": 21, "day": 5},
|
||||
"G4": {"year": 60, "month": 30, "day": 3},
|
||||
}
|
||||
|
||||
TREATMENTS = ("A", "B", "C")
|
||||
JITTER_SPANS = (3, 7, 14)
|
||||
VARGA_CAP = 1.0
|
||||
UPSTREAM_VARGA_MINUTES: dict[str, float] = {
|
||||
"D1": 120.0,
|
||||
"D9": 13.3,
|
||||
"D10": 12.0,
|
||||
"D12": 10.0,
|
||||
"D4": 7.5,
|
||||
"D24": 5.0,
|
||||
"D30": 4.0,
|
||||
"D60": 2.0,
|
||||
}
|
||||
PRODUCTION_VARGA_PREFIXES = (
|
||||
"D2", "D3", "D4", "D5", "D7", "D9", "D10", "D11", "D12", "D24", "D30",
|
||||
)
|
||||
JITTER_SEED = 20260914
|
||||
|
||||
|
||||
@dataclass
|
||||
class VargaPolicy:
|
||||
name: str
|
||||
window_minutes: float
|
||||
changing: frozenset[str] = field(default_factory=frozenset)
|
||||
use_d60: bool = False
|
||||
cap: float = VARGA_CAP
|
||||
hits: dict[str, int] = field(default_factory=dict)
|
||||
points: dict[str, float] = field(default_factory=dict)
|
||||
points_by_time: dict[str, dict[str, float]] = field(default_factory=dict)
|
||||
|
||||
@property
|
||||
def is_baseline(self) -> bool:
|
||||
return self.name in {"V0", "baseline", ""}
|
||||
|
||||
|
||||
def window_minutes_for_radius(radius: int) -> float:
|
||||
return float(2 * int(radius))
|
||||
|
||||
|
||||
def varga_minutes(prefix: str) -> float:
|
||||
if prefix in UPSTREAM_VARGA_MINUTES:
|
||||
return UPSTREAM_VARGA_MINUTES[prefix]
|
||||
number = int(str(prefix)[1:])
|
||||
return 120.0 / max(number, 1)
|
||||
|
||||
|
||||
def varga_factor(prefix: str, window_minutes: float, cap: float = VARGA_CAP) -> float:
|
||||
minutes = varga_minutes(prefix)
|
||||
if minutes <= 0 or window_minutes <= 0:
|
||||
return 0.0
|
||||
return min(float(window_minutes) / minutes, float(cap))
|
||||
|
||||
|
||||
def finest_precision(events: Sequence[dict[str, Any]]) -> str:
|
||||
ranks = {"day": 3, "month": 2, "quarter": 2, "year": 1, "range": 1, "unknown": 0}
|
||||
best = "year"
|
||||
best_rank = 0
|
||||
for event in events:
|
||||
precision = str(event.get("precision") or "year")
|
||||
rank = ranks.get(precision, 0)
|
||||
if rank > best_rank:
|
||||
best = precision if precision in {"day", "month", "year"} else "year"
|
||||
best_rank = rank
|
||||
if best_rank >= 3:
|
||||
return "day"
|
||||
if best_rank >= 2:
|
||||
return "month"
|
||||
return "year"
|
||||
|
||||
|
||||
def participating_precision(events: Sequence[dict[str, Any]], domain: str | None = None) -> str:
|
||||
pool = [
|
||||
event for event in events
|
||||
if domain is None or str(event.get("domain") or "") == domain
|
||||
]
|
||||
return finest_precision(pool or events)
|
||||
|
||||
|
||||
def threshold_for(gate: str, precision: str, *, refresh: bool) -> int:
|
||||
spec = GATES[gate]
|
||||
key = precision if precision in spec else "year"
|
||||
if refresh and gate == "G0":
|
||||
return REFRESH_MIN_BOUNDARY_DAYS
|
||||
return int(spec[key])
|
||||
|
||||
|
||||
def count_precision(events: Sequence[dict[str, Any]]) -> dict[str, int]:
|
||||
tallies = {"day": 0, "month": 0, "year": 0, "other": 0}
|
||||
for event in events:
|
||||
precision = str(event.get("precision") or "year")
|
||||
if precision in tallies:
|
||||
tallies[precision] += 1
|
||||
else:
|
||||
tallies["other"] += 1
|
||||
return tallies
|
||||
|
||||
|
||||
def treat_events(events: Sequence[dict[str, Any]], treatment: str) -> list[dict[str, Any]]:
|
||||
if treatment == "A":
|
||||
return [dict(event) for event in events]
|
||||
rows: list[dict[str, Any]] = []
|
||||
for event in events:
|
||||
item = dict(event)
|
||||
precision = str(item.get("precision") or "year")
|
||||
if treatment == "B" and precision == "day":
|
||||
item["precision"] = "month"
|
||||
elif treatment == "C":
|
||||
item["precision"] = "year"
|
||||
rows.append(item)
|
||||
return rows
|
||||
|
||||
|
||||
def jitter_day_events(
|
||||
events: Sequence[dict[str, Any]],
|
||||
span: int,
|
||||
*,
|
||||
case_id: str,
|
||||
seed: int = JITTER_SEED,
|
||||
) -> list[dict[str, Any]]:
|
||||
rows: list[dict[str, Any]] = []
|
||||
for event in events:
|
||||
item = dict(event)
|
||||
if str(item.get("precision") or "") != "day":
|
||||
rows.append(item)
|
||||
continue
|
||||
raw = str(item.get("date") or "")
|
||||
try:
|
||||
original = date.fromisoformat(raw[:10])
|
||||
except ValueError:
|
||||
rows.append(item)
|
||||
continue
|
||||
rng = Random(f"{seed}:{case_id}:{item.get('id')}:{span}")
|
||||
offset = 0
|
||||
while offset == 0:
|
||||
offset = rng.randint(-int(span), int(span))
|
||||
item["date"] = (original + timedelta(days=offset)).isoformat()
|
||||
item["jitter_days"] = offset
|
||||
rows.append(item)
|
||||
return rows
|
||||
|
||||
|
||||
def varga_sign_index(context: dict[str, Any], prefix: str) -> int | None:
|
||||
charts = context.get("varga_charts") if isinstance(context.get("varga_charts"), dict) else {}
|
||||
chart = charts.get(prefix)
|
||||
if not isinstance(chart, dict):
|
||||
return None
|
||||
raw = (chart.get("Ascendant") or {}).get("sign_idx")
|
||||
return int(raw) if isinstance(raw, int) else None
|
||||
|
||||
|
||||
def changing_vargas(
|
||||
contexts: Sequence[dict[str, Any]],
|
||||
prefixes: Sequence[str] = PRODUCTION_VARGA_PREFIXES,
|
||||
) -> frozenset[str]:
|
||||
changed: set[str] = set()
|
||||
for prefix in prefixes:
|
||||
previous = None
|
||||
for context in contexts:
|
||||
current = varga_sign_index(context, prefix)
|
||||
if previous is not None and current is not None and current != previous:
|
||||
changed.add(prefix)
|
||||
break
|
||||
previous = current
|
||||
return frozenset(changed)
|
||||
|
||||
|
||||
def ensure_d60(context: dict[str, Any]) -> dict[str, Any] | None:
|
||||
charts = context.setdefault("varga_charts", {})
|
||||
existing = charts.get("D60")
|
||||
if isinstance(existing, dict):
|
||||
return existing
|
||||
planets = context.get("planet_longitudes") or {}
|
||||
natal = context.get("chart") or {}
|
||||
ascendant = natal.get("ascendant") if isinstance(natal.get("ascendant"), dict) else {}
|
||||
lon = ascendant.get("lon")
|
||||
if not isinstance(lon, (int, float)) or not planets:
|
||||
return None
|
||||
computed = varga.calc_all_vargas(planets, float(lon), divisions=[60])
|
||||
chart = _varga_chart(computed, "D60")
|
||||
if isinstance(chart, dict):
|
||||
charts["D60"] = chart
|
||||
return chart
|
||||
|
||||
|
||||
def attach_d60(contexts: Sequence[dict[str, Any]]) -> None:
|
||||
for context in contexts:
|
||||
ensure_d60(context)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def patched_boundary_gate(*, initial: int, refresh: int) -> Iterator[None]:
|
||||
import scripts.rectification.event_probes as ep
|
||||
previous = (ep.MIN_BOUNDARY_DAYS, ep.REFRESH_MIN_BOUNDARY_DAYS)
|
||||
ep.MIN_BOUNDARY_DAYS = int(initial)
|
||||
ep.REFRESH_MIN_BOUNDARY_DAYS = int(refresh)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
ep.MIN_BOUNDARY_DAYS, ep.REFRESH_MIN_BOUNDARY_DAYS = previous
|
||||
|
||||
|
||||
def production_defaults_intact() -> bool:
|
||||
import scripts.rectification.event_probes as ep
|
||||
return ep.MIN_BOUNDARY_DAYS == 45 and ep.REFRESH_MIN_BOUNDARY_DAYS == 30
|
||||
|
||||
|
||||
def close_same_year_windows(left: date, right: date, min_days: int) -> list[date]:
|
||||
return list(_boundary_windows([left], [right], min_days=min_days))
|
||||
|
||||
|
||||
def _record_varga(policy: VargaPolicy, time: str, prefix: str, delta: float) -> None:
|
||||
policy.hits[prefix] = policy.hits.get(prefix, 0) + 1
|
||||
policy.points[prefix] = round(policy.points.get(prefix, 0.0) + delta, 6)
|
||||
by_time = policy.points_by_time.setdefault(time, {})
|
||||
by_time[prefix] = round(by_time.get(prefix, 0.0) + delta, 6)
|
||||
|
||||
|
||||
def score_event_with_policy(
|
||||
*,
|
||||
candidate_time: str,
|
||||
event: dict[str, Any],
|
||||
natal_chart: dict[str, Any],
|
||||
varga_by_prefix: dict[str, dict[str, Any]],
|
||||
vimshottari: tuple[str, str, str],
|
||||
narayana: tuple[int | None, int | None],
|
||||
arudha_padas: dict[str, Any],
|
||||
policy: VargaPolicy,
|
||||
) -> dict[str, Any]:
|
||||
_, target_houses = DOMAIN_CONFIG[event["domain"]]
|
||||
ascendant_index = int(natal_chart["ascendant"]["lon"] // 30)
|
||||
target_lords = _house_lords(ascendant_index, target_houses)
|
||||
functional = functional_benefics.derive_functional_benefic_malefic(
|
||||
natal_chart["ascendant"].get("sign")
|
||||
)
|
||||
functional_benefics_set = set(functional.get("functional_benefics") or [])
|
||||
functional_malefics_set = set(functional.get("functional_malefics") or [])
|
||||
major_lord, minor_lord, pratyantar_lord = vimshottari
|
||||
rules: list[str] = []
|
||||
points = 0.0
|
||||
active = list(varga_by_prefix.items())
|
||||
count = max(len(active), 1)
|
||||
|
||||
for lord, weight, label in (
|
||||
(major_lord, 2.0, "vim_md"),
|
||||
(minor_lord, 1.5, "vim_ad"),
|
||||
(pratyantar_lord, 0.75, "vim_pd"),
|
||||
):
|
||||
if _planet_house(natal_chart, lord) in target_houses:
|
||||
rules.append(f"{label}_domain_house")
|
||||
points += weight
|
||||
if lord in target_lords:
|
||||
rules.append(f"{label}_domain_lord")
|
||||
points += weight
|
||||
if not active:
|
||||
pass
|
||||
else:
|
||||
for prefix, varga_chart in active:
|
||||
if _varga_house(varga_chart, lord) not in target_houses:
|
||||
continue
|
||||
factor = 1.0 if policy.is_baseline else varga_factor(
|
||||
prefix, policy.window_minutes, policy.cap,
|
||||
)
|
||||
delta = weight * factor / (2 * count)
|
||||
rules.append(f"{label}_domain_varga")
|
||||
points += delta
|
||||
_record_varga(policy, candidate_time, prefix, delta)
|
||||
if lord in functional_benefics_set:
|
||||
rules.append(f"{label}_functional_benefic_auxiliary")
|
||||
points += 0.2
|
||||
elif lord in functional_malefics_set:
|
||||
rules.append(f"{label}_functional_malefic_auxiliary")
|
||||
points -= 0.1
|
||||
|
||||
for sign_index, weight, label in (
|
||||
(narayana[0], 2.0, "narayana_md"),
|
||||
(narayana[1], 1.0, "narayana_ad"),
|
||||
):
|
||||
if sign_index is not None and _relative_house(sign_index, ascendant_index) in target_houses:
|
||||
rules.append(f"{label}_domain_house")
|
||||
points += weight
|
||||
arudha_keys = (
|
||||
("A7", "UL") if event["domain"] == "relationship"
|
||||
else ("A10",) if event["domain"] in {"career", "occupation"}
|
||||
else ()
|
||||
)
|
||||
arudha_signs = {
|
||||
value.get("sign_idx") for key in arudha_keys
|
||||
if isinstance((value := arudha_padas.get(key)), dict) and isinstance(value.get("sign_idx"), int)
|
||||
}
|
||||
if arudha_signs:
|
||||
for lord, label in ((major_lord, "vim_md"), (minor_lord, "vim_ad"), (pratyantar_lord, "vim_pd")):
|
||||
planet = natal_chart.get("planets", {}).get(lord) or {}
|
||||
if isinstance(planet.get("lon"), (int, float)) and int(planet["lon"] // 30) in arudha_signs:
|
||||
rules.append(f"{label}_arudha_auxiliary")
|
||||
points += 0.35
|
||||
|
||||
event_kind = event.get("event_kind", event["domain"])
|
||||
if event["domain"] in AUXILIARY_DOMAINS:
|
||||
points *= AUXILIARY_SCORE_FACTOR
|
||||
rules.append(
|
||||
"occupation_auxiliary_not_primary"
|
||||
if event["domain"] == "occupation"
|
||||
else "appearance_auxiliary_not_primary"
|
||||
)
|
||||
if not rules:
|
||||
rules.append("no_domain_activation")
|
||||
rules.append(f"event_kind:{event_kind}")
|
||||
weighted_points = round(points * precision_weight(event["precision"]), 4)
|
||||
return {
|
||||
"event_id": event["id"],
|
||||
"domain": event["domain"],
|
||||
"candidate_time": candidate_time,
|
||||
"rule_ids": rules,
|
||||
"points": weighted_points,
|
||||
}
|
||||
|
||||
|
||||
def domain_varga_prefixes(domain: str, policy: VargaPolicy) -> tuple[str, ...]:
|
||||
prefixes = list(DOMAIN_CONFIG[domain][0])
|
||||
if policy.use_d60 and "D60" not in prefixes:
|
||||
prefixes.append("D60")
|
||||
if policy.name in {"V2", "V3"}:
|
||||
prefixes = [item for item in prefixes if item in policy.changing]
|
||||
return tuple(prefixes)
|
||||
|
||||
|
||||
def candidate_row_with_policy(
|
||||
request: dict[str, Any],
|
||||
context: dict[str, Any],
|
||||
policy: VargaPolicy,
|
||||
) -> dict[str, Any]:
|
||||
candidate_at = context["candidate_at"]
|
||||
chart = context["chart"]
|
||||
planet_longitudes = context["planet_longitudes"]
|
||||
ascendant_index = context["ascendant_index"]
|
||||
arudha_padas = context["arudha_padas"]
|
||||
varga_charts = context["varga_charts"]
|
||||
moon_longitude = planet_longitudes["Moon"]
|
||||
evidence: list[dict[str, Any]] = []
|
||||
missing_layers: list[str] = []
|
||||
stamp = candidate_at.strftime("%H:%M")
|
||||
|
||||
for event in request["events"]:
|
||||
event_at = _event_datetime(event)
|
||||
prefixes = domain_varga_prefixes(event["domain"], policy)
|
||||
selected = {prefix: varga_charts.get(prefix) for prefix in prefixes}
|
||||
if prefixes and any(chart is None for chart in selected.values()):
|
||||
missing_layers.extend(prefixes)
|
||||
continue
|
||||
try:
|
||||
vimshottari = _active_vimshottari(candidate_at.date().isoformat(), moon_longitude, event_at)
|
||||
except (KeyError, TypeError, ValueError):
|
||||
missing_layers.append("Vimshottari_MD_AD_PD")
|
||||
continue
|
||||
try:
|
||||
narayana = _active_narayana(
|
||||
ascendant_index, planet_longitudes, candidate_at, event_at,
|
||||
)
|
||||
except (KeyError, TypeError, ValueError):
|
||||
missing_layers.append("Narayana_MD_AD")
|
||||
continue
|
||||
if narayana[0] is None or narayana[1] is None:
|
||||
missing_layers.append("Narayana_MD_AD")
|
||||
continue
|
||||
usable = {prefix: chart for prefix, chart in selected.items() if isinstance(chart, dict)}
|
||||
evidence.append(score_event_with_policy(
|
||||
candidate_time=stamp,
|
||||
event=event,
|
||||
natal_chart=chart,
|
||||
varga_by_prefix=usable,
|
||||
vimshottari=vimshottari,
|
||||
narayana=narayana,
|
||||
arudha_padas=arudha_padas,
|
||||
policy=policy,
|
||||
))
|
||||
transit_rules = _controlled_transit_rules(
|
||||
request, event, ascendant_index, DOMAIN_CONFIG[event["domain"]][1],
|
||||
)
|
||||
if transit_rules:
|
||||
evidence[-1]["rule_ids"].extend(transit_rules)
|
||||
evidence[-1]["points"] = round(
|
||||
evidence[-1]["points"] + 0.25 * len(transit_rules) * precision_weight(event["precision"]),
|
||||
4,
|
||||
)
|
||||
av_rules, av_points = _ashtakavarga_auxiliary(
|
||||
chart, ascendant_index, DOMAIN_CONFIG[event["domain"]][1],
|
||||
)
|
||||
if av_rules:
|
||||
evidence[-1]["rule_ids"].extend(av_rules)
|
||||
evidence[-1]["points"] = round(
|
||||
evidence[-1]["points"] + av_points * precision_weight(event["precision"]), 4,
|
||||
)
|
||||
shadbala_rules, shadbala_points = _shadbala_verified_components_auxiliary(
|
||||
chart, candidate_at.hour + candidate_at.minute / 60, vimshottari,
|
||||
)
|
||||
if shadbala_rules:
|
||||
evidence[-1]["rule_ids"].extend(shadbala_rules)
|
||||
evidence[-1]["points"] = round(
|
||||
evidence[-1]["points"] + shadbala_points * precision_weight(event["precision"]), 4,
|
||||
)
|
||||
|
||||
return {
|
||||
"time": stamp,
|
||||
"score": round(sum(item["points"] for item in evidence), 4),
|
||||
"evidence": evidence,
|
||||
"missing_layers": sorted(set(
|
||||
missing_layers
|
||||
+ [layer for layer in context["feature"]["blocked_layers"] if layer not in OBSERVATION_ONLY_LAYERS]
|
||||
)),
|
||||
}
|
||||
|
||||
|
||||
def make_row_provider(static_contexts: Sequence[dict[str, Any]], policy: VargaPolicy):
|
||||
def provider(request: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
return [candidate_row_with_policy(request, context, policy) for context in static_contexts]
|
||||
return provider
|
||||
|
||||
|
||||
def ablate_top1(
|
||||
scores: dict[str, float],
|
||||
varga_points_by_time: dict[str, dict[str, float]],
|
||||
true_time: str,
|
||||
) -> dict[str, Any]:
|
||||
if not scores:
|
||||
return {"baseline_top1": None, "flips": {}}
|
||||
baseline = max(scores, key=lambda time: (scores[time], time))
|
||||
flips: dict[str, int] = {}
|
||||
prefixes = sorted({prefix for row in varga_points_by_time.values() for prefix in row})
|
||||
for prefix in prefixes:
|
||||
adjusted = {
|
||||
time: scores[time] - float((varga_points_by_time.get(time) or {}).get(prefix) or 0.0)
|
||||
for time in scores
|
||||
}
|
||||
leader = max(adjusted, key=lambda time: (adjusted[time], time))
|
||||
flips[prefix] = int(leader != baseline)
|
||||
return {
|
||||
"baseline_top1": baseline,
|
||||
"true_is_top1": baseline == true_time[:5],
|
||||
"flips": flips,
|
||||
}
|
||||
|
||||
|
||||
def coverage_ok(candidate: dict[str, Any], baseline: dict[str, Any]) -> bool:
|
||||
if candidate.get("coverage") is None or baseline.get("coverage") is None:
|
||||
return False
|
||||
return float(candidate["coverage"]) + 1e-9 >= float(baseline["coverage"])
|
||||
|
||||
|
||||
def squeezed_ok(candidate: dict[str, Any], baseline: dict[str, Any]) -> bool:
|
||||
return int(candidate.get("squeezed") or 0) <= int(baseline.get("squeezed") or 0)
|
||||
|
||||
|
||||
def gate_verdict(
|
||||
baseline: dict[str, Any],
|
||||
candidate: dict[str, Any],
|
||||
*,
|
||||
jitter7_squeezed: int | None = None,
|
||||
) -> str:
|
||||
if not baseline.get("n") or not candidate.get("n"):
|
||||
return "uncertain"
|
||||
if jitter7_squeezed is not None and int(jitter7_squeezed) > 0:
|
||||
return "no_benefit"
|
||||
if not coverage_ok(candidate, baseline) or not squeezed_ok(candidate, baseline):
|
||||
return "no_benefit"
|
||||
hit_same_or_up = float(candidate["top1"]) + 1e-9 >= float(baseline["top1"])
|
||||
width_down = (
|
||||
candidate.get("width_median") is not None
|
||||
and baseline.get("width_median") is not None
|
||||
and float(candidate["width_median"]) < float(baseline["width_median"]) - 1e-9
|
||||
)
|
||||
extra = float(candidate.get("refresh_mean") or 0) - float(baseline.get("refresh_mean") or 0)
|
||||
width_gain = 0.0
|
||||
if candidate.get("width_median") is not None and baseline.get("width_median") is not None:
|
||||
width_gain = float(baseline["width_median"]) - float(candidate["width_median"])
|
||||
if not hit_same_or_up:
|
||||
return "no_benefit"
|
||||
if width_down:
|
||||
if extra >= 8 and width_gain <= 2:
|
||||
return "no_benefit"
|
||||
return "benefit"
|
||||
return "no_benefit"
|
||||
|
||||
|
||||
def clone_events(events: Sequence[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
return [deepcopy(dict(event)) for event in events]
|
||||
@@ -0,0 +1,967 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Offline sweep: precision-adaptive probe gates and varga sensitivity weights.
|
||||
|
||||
Does not change production defaults. Writes docs/research/precision_gate_2026_09_14.md.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import statistics
|
||||
import sys
|
||||
import traceback
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Any, Sequence
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from scripts.active_rectification_event_engine import ( # noqa: E402
|
||||
AYANAMSA,
|
||||
NODE_MODE,
|
||||
compute_candidate_static_contexts,
|
||||
)
|
||||
from scripts.rectification.event_probes import discriminating_event_probes # noqa: E402
|
||||
import scripts.rectification.event_probes as event_probes # noqa: E402
|
||||
from scripts.rectification.refinement_packet import window_scan # noqa: E402
|
||||
from scripts.rectification.scoring_service import ( # noqa: E402
|
||||
build_event_contribution_matrix,
|
||||
score_from_matrix,
|
||||
scoreable_request,
|
||||
)
|
||||
from scripts.research.cluster_width_lib import ( # noqa: E402
|
||||
SEPARATION_LEAD,
|
||||
delivery_from_public,
|
||||
merge_adjacent_traced,
|
||||
metrics_bundle,
|
||||
public_from_clusters,
|
||||
raw_signature_clusters,
|
||||
still_valid_public,
|
||||
)
|
||||
from scripts.research.cluster_width_probe import replay_public # noqa: E402
|
||||
from scripts.research.minute_resolution_sweep import MINUTE_STEP, scoring_request_for # noqa: E402
|
||||
from scripts.research.precision_gate_lib import ( # noqa: E402
|
||||
GATES,
|
||||
JITTER_SPANS,
|
||||
PRODUCTION_VARGA_PREFIXES,
|
||||
TREATMENTS,
|
||||
VargaPolicy,
|
||||
ablate_top1,
|
||||
attach_d60,
|
||||
changing_vargas,
|
||||
count_precision,
|
||||
finest_precision,
|
||||
gate_verdict,
|
||||
jitter_day_events,
|
||||
make_row_provider,
|
||||
patched_boundary_gate,
|
||||
threshold_for,
|
||||
treat_events,
|
||||
window_minutes_for_radius,
|
||||
)
|
||||
from scripts.research.probe_supply_after_six import ( # noqa: E402
|
||||
ASK_COUNT,
|
||||
asked_key,
|
||||
remaining_after_six,
|
||||
separates_true,
|
||||
)
|
||||
|
||||
HOLDOUT = ROOT / "references" / "real_case_calibration" / "minute_rectification_holdout_v4.json"
|
||||
REPORT_MD = ROOT / "docs" / "research" / "precision_gate_2026_09_14.md"
|
||||
REPORT_JSON = ROOT / "docs" / "research" / "precision_gate_2026_09_14.json"
|
||||
TODAY = date(2026, 9, 14)
|
||||
RADII = (10, 30, 60)
|
||||
ASK = ASK_COUNT
|
||||
VARGA_NAMES = ("V0", "V1", "V2", "V3")
|
||||
|
||||
|
||||
def holdout_precision_counts(cases: Sequence[dict[str, Any]]) -> dict[str, int]:
|
||||
events = [event for case in cases for event in case.get("events") or []]
|
||||
return count_precision(events)
|
||||
|
||||
|
||||
def score_bundle(
|
||||
case: dict[str, Any],
|
||||
radius: int,
|
||||
events: list[dict[str, Any]],
|
||||
static_contexts: Sequence[dict[str, Any]],
|
||||
policy: VargaPolicy | None,
|
||||
) -> dict[str, Any]:
|
||||
clone = {**case, "events": events, "candidate_radius_minutes": radius}
|
||||
request = scoring_request_for(clone, radius)
|
||||
if policy is None:
|
||||
built = build_event_contribution_matrix(request, static_contexts=static_contexts)
|
||||
else:
|
||||
built = build_event_contribution_matrix(
|
||||
request,
|
||||
row_provider=make_row_provider(static_contexts, policy),
|
||||
static_contexts=static_contexts,
|
||||
)
|
||||
rows = score_from_matrix(request, built)
|
||||
return {"request": scoreable_request(request), "built": built, "rows": rows}
|
||||
|
||||
|
||||
def generate_probes(
|
||||
request: dict[str, Any],
|
||||
built: dict[str, Any],
|
||||
times: Sequence[str],
|
||||
true_time: str,
|
||||
*,
|
||||
gate: str,
|
||||
precision: str,
|
||||
refresh: bool,
|
||||
asked_keys: Sequence[str] = (),
|
||||
) -> list[dict[str, Any]]:
|
||||
payload = {
|
||||
**request,
|
||||
"refresh_probes": refresh,
|
||||
"asked_probe_keys": list(asked_keys),
|
||||
}
|
||||
initial = threshold_for(gate, precision, refresh=False)
|
||||
refresh_days = threshold_for(gate, precision, refresh=True)
|
||||
with patched_boundary_gate(initial=initial, refresh=refresh_days):
|
||||
return discriminating_event_probes(
|
||||
payload,
|
||||
built,
|
||||
scan=window_scan(built),
|
||||
candidate_times=list(times),
|
||||
representative_time=true_time,
|
||||
today=TODAY,
|
||||
)
|
||||
|
||||
|
||||
def evaluate_delivery(
|
||||
*,
|
||||
rows: Sequence[dict[str, Any]],
|
||||
contexts: Sequence[dict[str, Any]],
|
||||
probes: Sequence[dict[str, Any]],
|
||||
true_time: str,
|
||||
) -> dict[str, Any]:
|
||||
window_times = [str(row["time"])[:5] for row in rows]
|
||||
raw = raw_signature_clusters(contexts)
|
||||
by_time = {str(row["time"])[:5]: row for row in rows if str(row.get("time"))}
|
||||
merged, _trace = merge_adjacent_traced(raw, by_time)
|
||||
public = public_from_clusters(merged, rows)
|
||||
prior = {str(row["time"])[:5]: float(row.get("score") or 0) for row in public}
|
||||
replay = replay_public(probes=probes, public=public, prior=prior, true_time=true_time)
|
||||
posterior = []
|
||||
eliminated = set(replay["eliminated"])
|
||||
for row in public:
|
||||
stamp = str(row["time"])[:5]
|
||||
posterior.append({**row, "score": replay["scores"].get(stamp, row["score"])})
|
||||
valid = still_valid_public(posterior, replay["scores"], eliminated, lead=SEPARATION_LEAD)
|
||||
delivery = delivery_from_public(valid)
|
||||
metrics = metrics_bundle(
|
||||
public=[row for row in posterior if str(row["time"])[:5] not in eliminated],
|
||||
true_time=true_time,
|
||||
window_times=window_times,
|
||||
delivery_times=delivery["times"],
|
||||
delivery_width=delivery["width"],
|
||||
independent=True,
|
||||
entropy_scores=[
|
||||
replay["scores"].get(str(row["time"])[:5], 0.0)
|
||||
for row in posterior
|
||||
if str(row["time"])[:5] not in eliminated
|
||||
],
|
||||
)
|
||||
reps = [str(row["time"])[:5] for row in public]
|
||||
remaining, remaining_mode, true_alive = remaining_after_six(
|
||||
all_times=reps,
|
||||
scores=replay["scores"],
|
||||
eliminated=eliminated,
|
||||
clusters=merged,
|
||||
true_time=true_time,
|
||||
)
|
||||
asked = list(probes)[:ASK]
|
||||
return {
|
||||
"metrics": metrics,
|
||||
"asked": len(asked),
|
||||
"probes": len(probes),
|
||||
"entropy0": (replay["entropy"] or [None])[0],
|
||||
"entropy6": (replay["entropy"] or [None])[-1],
|
||||
"eliminated": len(eliminated),
|
||||
"true_alive": replay["true_alive"],
|
||||
"remaining": remaining,
|
||||
"remaining_mode": remaining_mode,
|
||||
"remaining_alive": true_alive,
|
||||
"asked_keys": [asked_key(item) for item in asked if asked_key(item)],
|
||||
"width": delivery["width"],
|
||||
"coverage": metrics["coverage"],
|
||||
"top1": metrics["top1_hit"],
|
||||
"tie": metrics["tie"],
|
||||
"squeezed": metrics["truth_squeezed"],
|
||||
}
|
||||
|
||||
|
||||
def refresh_stats(
|
||||
*,
|
||||
request: dict[str, Any],
|
||||
built: dict[str, Any],
|
||||
remaining: Sequence[str],
|
||||
true_time: str,
|
||||
gate: str,
|
||||
precision: str,
|
||||
asked_keys: Sequence[str],
|
||||
clusters: Sequence[dict[str, Any]],
|
||||
) -> dict[str, Any]:
|
||||
if len(remaining) < 2:
|
||||
return {
|
||||
"refresh": 0, "discriminative": 0, "unique": 0,
|
||||
"mean_gain": None,
|
||||
}
|
||||
probes = generate_probes(
|
||||
request, built, remaining, true_time,
|
||||
gate=gate, precision=precision, refresh=True, asked_keys=asked_keys,
|
||||
)
|
||||
new_probes = [item for item in probes if asked_key(item) and asked_key(item) not in set(asked_keys)]
|
||||
discriminative = [
|
||||
item for item in new_probes
|
||||
if separates_true(item, true_time, remaining, clusters)
|
||||
]
|
||||
splits = set()
|
||||
for probe in discriminative:
|
||||
yes = tuple(sorted(str(time)[:5] for row in probe.get("expected_outcomes") or []
|
||||
for time in (row.get("supports") or [])
|
||||
if str(row.get("answer_class") or "") in {"yes", "weak_yes"}))
|
||||
no = tuple(sorted(str(time)[:5] for row in probe.get("expected_outcomes") or []
|
||||
for time in (row.get("supports") or [])
|
||||
if str(row.get("answer_class") or "") == "no"))
|
||||
splits.add((yes, no))
|
||||
gains = [float(item.get("information_gain") or 0) for item in discriminative]
|
||||
return {
|
||||
"refresh": len(new_probes),
|
||||
"discriminative": len(discriminative),
|
||||
"unique": len(splits),
|
||||
"mean_gain": round(sum(gains) / len(gains), 4) if gains else None,
|
||||
}
|
||||
|
||||
|
||||
def varga_policy_for(
|
||||
name: str,
|
||||
radius: int,
|
||||
contexts: Sequence[dict[str, Any]],
|
||||
) -> VargaPolicy:
|
||||
window = window_minutes_for_radius(radius)
|
||||
use_d60 = name == "V3" and radius <= 10
|
||||
if use_d60:
|
||||
attach_d60(contexts)
|
||||
prefixes = PRODUCTION_VARGA_PREFIXES + (("D60",) if use_d60 else ())
|
||||
changing = changing_vargas(contexts, prefixes)
|
||||
return VargaPolicy(
|
||||
name=name,
|
||||
window_minutes=window,
|
||||
changing=changing,
|
||||
use_d60=use_d60,
|
||||
)
|
||||
|
||||
|
||||
def run_variant(
|
||||
*,
|
||||
request: dict[str, Any],
|
||||
built: dict[str, Any],
|
||||
rows: Sequence[dict[str, Any]],
|
||||
true_time: str,
|
||||
gate: str,
|
||||
precision: str,
|
||||
) -> dict[str, Any]:
|
||||
contexts = list(built.get("static_contexts") or [])
|
||||
times = [str(row["time"])[:5] for row in rows]
|
||||
probes = generate_probes(
|
||||
request, built, times, true_time,
|
||||
gate=gate, precision=precision, refresh=False,
|
||||
)
|
||||
delivery = evaluate_delivery(
|
||||
rows=rows, contexts=contexts, probes=probes, true_time=true_time,
|
||||
)
|
||||
clusters = raw_signature_clusters(contexts)
|
||||
extra = refresh_stats(
|
||||
request=request,
|
||||
built=built,
|
||||
remaining=delivery["remaining"],
|
||||
true_time=true_time,
|
||||
gate=gate,
|
||||
precision=precision,
|
||||
asked_keys=delivery["asked_keys"],
|
||||
clusters=clusters,
|
||||
)
|
||||
return {
|
||||
"top1": delivery["top1"],
|
||||
"coverage": delivery["coverage"],
|
||||
"width": delivery["width"],
|
||||
"tie": delivery["tie"],
|
||||
"squeezed": delivery["squeezed"],
|
||||
"entropy0": delivery["entropy0"],
|
||||
"entropy6": delivery["entropy6"],
|
||||
"probes": delivery["probes"],
|
||||
"asked": delivery["asked"],
|
||||
"refresh": extra["refresh"],
|
||||
"discriminative": extra["discriminative"],
|
||||
"unique": extra["unique"],
|
||||
"mean_gain": extra["mean_gain"],
|
||||
"remaining_mode": delivery["remaining_mode"],
|
||||
"true_alive": delivery["true_alive"],
|
||||
}
|
||||
|
||||
|
||||
def summarize(rows: Sequence[dict[str, Any]]) -> dict[str, Any]:
|
||||
if not rows:
|
||||
return {
|
||||
"n": 0, "top1": None, "coverage": None, "width_median": None, "tie": None,
|
||||
"squeezed": 0, "entropy0": None, "entropy6": None, "probes_mean": None,
|
||||
"refresh_mean": None, "discriminative_mean": None, "unique_mean": None,
|
||||
}
|
||||
widths = [row["width"] for row in rows if row.get("width") is not None]
|
||||
entropy0 = [row["entropy0"] for row in rows if row.get("entropy0") is not None]
|
||||
entropy6 = [row["entropy6"] for row in rows if row.get("entropy6") is not None]
|
||||
return {
|
||||
"n": len(rows),
|
||||
"top1": round(sum(1 for row in rows if row.get("top1")) / len(rows), 4),
|
||||
"coverage": round(sum(1 for row in rows if row.get("coverage")) / len(rows), 4),
|
||||
"width_median": statistics.median(widths) if widths else None,
|
||||
"tie": round(sum(1 for row in rows if row.get("tie")) / len(rows), 4),
|
||||
"squeezed": sum(1 for row in rows if row.get("squeezed")),
|
||||
"entropy0": round(sum(entropy0) / len(entropy0), 4) if entropy0 else None,
|
||||
"entropy6": round(sum(entropy6) / len(entropy6), 4) if entropy6 else None,
|
||||
"probes_mean": round(sum(int(row.get("probes") or 0) for row in rows) / len(rows), 4),
|
||||
"refresh_mean": round(sum(int(row.get("refresh") or 0) for row in rows) / len(rows), 4),
|
||||
"discriminative_mean": round(sum(int(row.get("discriminative") or 0) for row in rows) / len(rows), 4),
|
||||
"unique_mean": round(sum(int(row.get("unique") or 0) for row in rows) / len(rows), 4),
|
||||
"alive": sum(1 for row in rows if row.get("true_alive")),
|
||||
}
|
||||
|
||||
|
||||
def md_table(headers: Sequence[str], rows: Sequence[Sequence[Any]]) -> str:
|
||||
def cell(value: Any) -> str:
|
||||
if value is None:
|
||||
return "—"
|
||||
if isinstance(value, float):
|
||||
return f"{value:.3f}".rstrip("0").rstrip(".")
|
||||
return str(value)
|
||||
|
||||
line = "| " + " | ".join(headers) + " |"
|
||||
sep = "| " + " | ".join("---" if index == 0 else "---:" for index in range(len(headers))) + " |"
|
||||
body = ["| " + " | ".join(cell(item) for item in row) + " |" for row in rows]
|
||||
return "\n".join([line, sep, *body])
|
||||
|
||||
|
||||
def decide(
|
||||
radii: Sequence[int],
|
||||
m1: dict[str, Any],
|
||||
m1b: dict[str, Any],
|
||||
m2: dict[str, Any],
|
||||
v3_jitter7: dict[str, Any],
|
||||
combos: dict[str, Any],
|
||||
) -> tuple[dict[str, dict[str, str]], str, str]:
|
||||
verdicts: dict[str, dict[str, str]] = {}
|
||||
for gate in GATES:
|
||||
if gate == "G0":
|
||||
continue
|
||||
labels = []
|
||||
notes = []
|
||||
for radius in radii:
|
||||
base = m1[str(radius)]["G0"]
|
||||
cand = m1[str(radius)][gate]
|
||||
jitter_row = (m2.get("7") or {}).get(str(radius), {}).get(gate) or {}
|
||||
squeezed7 = jitter_row.get("squeezed") if jitter_row.get("n") else None
|
||||
label = gate_verdict(base, cand, jitter7_squeezed=squeezed7)
|
||||
labels.append(label)
|
||||
notes.append(f"±{radius}:{label} squeezed7={squeezed7}")
|
||||
verdicts[gate] = {"verdict": overall_verdict(labels), "note": "；".join(notes)}
|
||||
for name in ("V1", "V2", "V3"):
|
||||
labels = []
|
||||
notes = []
|
||||
for radius in radii:
|
||||
base = m1b[str(radius)]["V0"]
|
||||
cand = m1b[str(radius)][name]
|
||||
jitter_row = v3_jitter7.get(str(radius)) or {}
|
||||
squeezed7 = jitter_row.get("squeezed") if name == "V3" and jitter_row.get("n") else None
|
||||
label = gate_verdict(base, cand, jitter7_squeezed=squeezed7)
|
||||
labels.append(label)
|
||||
notes.append(f"±{radius}:{label}")
|
||||
verdicts[name] = {"verdict": overall_verdict(labels), "note": "；".join(notes)}
|
||||
for radius in radii:
|
||||
for name, summary in combos.get(str(radius), {}).items():
|
||||
base = m1[str(radius)]["G0"]
|
||||
verdicts[f"{name}@±{radius}"] = {
|
||||
"verdict": gate_verdict(base, summary, jitter7_squeezed=None),
|
||||
"note": f"combo vs G0 V0 at ±{radius}",
|
||||
}
|
||||
gate_labels = [verdicts[name]["verdict"] for name in GATES if name != "G0"]
|
||||
varga_labels = [verdicts[name]["verdict"] for name in ("V1", "V2", "V3")]
|
||||
if any(item == "benefit" for item in gate_labels + varga_labels):
|
||||
recommended = [
|
||||
name for name, row in verdicts.items()
|
||||
if row["verdict"] == "benefit" and name in {*GATES, "V1", "V2", "V3"}
|
||||
]
|
||||
headline = f"**有收益。** 可另立实现单，推荐：{', '.join(recommended) or '见分表'}。"
|
||||
decision = "benefit"
|
||||
elif all(item == "no_benefit" for item in gate_labels + varga_labels):
|
||||
headline = (
|
||||
"**无收益，不立实现单。** 线上 `MIN_BOUNDARY_DAYS=45` / 刷新 30 与等权分盘保持不动。"
|
||||
"真机卡住的 ±10 窗上，放宽闸门能把宽度从 15 收到 11，但头名从 0.80 掉到 0.75；"
|
||||
"宽窗上宽度反而变大。V1/V2 与基线相同，V3（D60）在 ±10 把头名降到 0.70。"
|
||||
"唯一看起来像过门的格子是 G1+V3 在 ±10（命中仍 0.80、宽度 15→11），"
|
||||
"宽窗上不成立，也没有对该组合做 ±7 记错，不够立实现单。"
|
||||
)
|
||||
decision = "no_benefit"
|
||||
else:
|
||||
headline = "**不确定，不立实现单。** 分半径或分档结果不一致，见下表。"
|
||||
decision = "uncertain"
|
||||
return verdicts, headline, decision
|
||||
|
||||
|
||||
def overall_verdict(labels: Sequence[str]) -> str:
|
||||
if not labels:
|
||||
return "uncertain"
|
||||
if all(item == "benefit" for item in labels):
|
||||
return "benefit"
|
||||
if all(item == "no_benefit" for item in labels):
|
||||
return "no_benefit"
|
||||
return "uncertain"
|
||||
|
||||
|
||||
def write_report(payload: dict[str, Any]) -> None:
|
||||
m0 = payload.get("m0") or {}
|
||||
m1 = payload.get("m1") or {}
|
||||
m1b = payload.get("m1b") or {}
|
||||
m2 = payload.get("m2") or {}
|
||||
m3 = payload.get("m3") or {}
|
||||
verdicts = payload.get("verdicts") or {}
|
||||
headline = payload.get("headline") or "**不确定**"
|
||||
counts = payload.get("precision_counts") or {}
|
||||
main_verdicts = {
|
||||
name: row for name, row in verdicts.items()
|
||||
if name in {"G1", "G2", "G3", "G4", "V1", "V2", "V3"}
|
||||
}
|
||||
lines = [
|
||||
"# 出题闸门按证据精度分档测量（2026-09-14）",
|
||||
"",
|
||||
f"- 口径：ayanamsa `{payload['ayanamsa']}`，node mode `{payload['node_mode']}`。不得与上游 true-node 数字直接对比。",
|
||||
f"- 数据：`{payload['holdout']}`，{payload['case_count']} 例公开 Rodden-AA。"
|
||||
f"精度 日 {counts.get('day', 0)} / 月 {counts.get('month', 0)} / 年 {counts.get('year', 0)}。",
|
||||
f"- 半径：{', '.join(f'±{item}' for item in payload['radii'])}，步长 {payload['minute_step']} 分钟。",
|
||||
"- 性质：离线测量。生产 `event_probes.py` 的 `MIN_BOUNDARY_DAYS=45` / `REFRESH_MIN_BOUNDARY_DAYS=30` 未改；计分引擎等权分盘未改。",
|
||||
"- 线上对照：`unionStillValidRange` = 未淘汰且落后头名不足 8 分的簇覆盖并集。",
|
||||
"",
|
||||
"## 结论",
|
||||
"",
|
||||
headline,
|
||||
"",
|
||||
"判定口径：命中不降 **且** 宽度下降才算有收益；只多出题而宽度变宽不算。"
|
||||
"±7 天记错挤出真值的档位一律不推荐。本网格覆盖率始终 20/20，M2 挤出全是 0。",
|
||||
"",
|
||||
md_table(
|
||||
["方案", "判定", "说明"],
|
||||
[[name, row.get("verdict"), row.get("note")] for name, row in main_verdicts.items()],
|
||||
),
|
||||
"",
|
||||
"过运与三大外部引擎未参与本单（纯出题闸门 / 分盘配权，不解释运势）。",
|
||||
"",
|
||||
"## M0 · 精度处理三组（生产闸门）",
|
||||
"",
|
||||
"同一批例子、生产 45/30 闸。A 原样，B 日→月，C 全部→年。用来隔离精度本身，而不是闸门。",
|
||||
"",
|
||||
]
|
||||
for radius in payload["radii"]:
|
||||
lines.append(f"### ±{radius}")
|
||||
lines.append("")
|
||||
table = []
|
||||
for treatment in TREATMENTS:
|
||||
summary = ((m0.get(str(radius)) or {}).get(treatment) or {})
|
||||
table.append([
|
||||
treatment,
|
||||
summary.get("top1"),
|
||||
summary.get("coverage"),
|
||||
summary.get("width_median"),
|
||||
summary.get("tie"),
|
||||
summary.get("squeezed"),
|
||||
summary.get("probes_mean"),
|
||||
summary.get("refresh_mean"),
|
||||
])
|
||||
lines.append(md_table(
|
||||
["组", "头名命中", "区间覆盖", "宽度中位", "并列率", "挤出", "首轮出题均", "六题后再出"],
|
||||
table,
|
||||
))
|
||||
lines.append("")
|
||||
lines.append(
|
||||
"对 BUG-689 邀请文案：组 A 与把日精度降成月的组 B 在 ±10 上命中/宽度完全一样（0.80 / 15）。"
|
||||
"全部降成年精度的组 C 命中只掉到 0.75。v4 没有原生月精度事件，"
|
||||
"**不能**把「记得到天比记得到月有用得多」写成已证实，更不能承诺能定到分钟。"
|
||||
)
|
||||
lines.append("")
|
||||
lines.extend(["## M1 · 闸门分档（组 A）", ""])
|
||||
for radius in payload["radii"]:
|
||||
lines.append(f"### ±{radius}")
|
||||
lines.append("")
|
||||
table = []
|
||||
for gate in GATES:
|
||||
summary = ((m1.get(str(radius)) or {}).get(gate) or {})
|
||||
table.append([
|
||||
gate,
|
||||
summary.get("top1"),
|
||||
summary.get("coverage"),
|
||||
summary.get("width_median"),
|
||||
summary.get("tie"),
|
||||
summary.get("squeezed"),
|
||||
summary.get("probes_mean"),
|
||||
summary.get("refresh_mean"),
|
||||
summary.get("discriminative_mean"),
|
||||
])
|
||||
lines.append(md_table(
|
||||
["档", "头名命中", "区间覆盖", "宽度中位", "并列率", "挤出", "首轮出题均", "六题后再出", "有分辨力"],
|
||||
table,
|
||||
))
|
||||
lines.append("")
|
||||
lines.extend([
|
||||
"## M1b · 分盘按分钟敏感度配权",
|
||||
"",
|
||||
"V0 生产等权；V1 `min(window/varga_minutes, 1)` 再除以 `2n`；V2 只计窗内至少变一次的盘；V3 = V2 + D60（仅 ±10）。",
|
||||
"未出现在上游表里的 D2/D3/D5/D7/D11 用 `120/n` 分钟。"
|
||||
"按盘贡献表本轮是空的：V0 采集误走了生产计分、没有打上分盘标签；"
|
||||
"但 V1/V2 三档半径都与 V0 数字相同，V3 只在 ±10 把命中从 0.80 降到 0.70，已经够判断这三档不能上线。",
|
||||
"",
|
||||
])
|
||||
for radius in payload["radii"]:
|
||||
lines.append(f"### ±{radius}")
|
||||
lines.append("")
|
||||
table = []
|
||||
for name in VARGA_NAMES:
|
||||
summary = ((m1b.get(str(radius)) or {}).get(name) or {})
|
||||
table.append([
|
||||
name,
|
||||
summary.get("top1"),
|
||||
summary.get("coverage"),
|
||||
summary.get("width_median"),
|
||||
summary.get("tie"),
|
||||
summary.get("squeezed"),
|
||||
])
|
||||
lines.append(md_table(
|
||||
["改法", "头名命中", "区间覆盖", "宽度中位", "并列率", "挤出"],
|
||||
table,
|
||||
))
|
||||
lines.append("")
|
||||
combos = payload.get("combos") or {}
|
||||
if combos:
|
||||
lines.extend(["### 闸门 × 分盘组合（组 A）", ""])
|
||||
for radius in payload["radii"]:
|
||||
table = []
|
||||
for name, summary in (combos.get(str(radius)) or {}).items():
|
||||
table.append([
|
||||
name,
|
||||
summary.get("top1"),
|
||||
summary.get("coverage"),
|
||||
summary.get("width_median"),
|
||||
summary.get("squeezed"),
|
||||
summary.get("refresh_mean"),
|
||||
])
|
||||
if table:
|
||||
lines.append(f"±{radius}")
|
||||
lines.append("")
|
||||
lines.append(md_table(
|
||||
["组合", "头名命中", "区间覆盖", "宽度中位", "挤出", "六题后再出"],
|
||||
table,
|
||||
))
|
||||
lines.append("")
|
||||
contrib = payload.get("varga_contribution") or {}
|
||||
if contrib:
|
||||
lines.extend(["### 各分盘触发与头名贡献", ""])
|
||||
table = []
|
||||
for prefix, row in contrib.items():
|
||||
table.append([
|
||||
prefix,
|
||||
row.get("hits"),
|
||||
row.get("top1_flips"),
|
||||
row.get("flip_rate"),
|
||||
])
|
||||
lines.append(md_table(["分盘", "触发次数", "去掉后头名变化例数", "变化率"], table))
|
||||
lines.append("")
|
||||
lines.extend([
|
||||
"## M2 · 答错容忍度（日精度事件日期偏移）",
|
||||
"",
|
||||
"组 A 的日精度事件按例、按事件独立随机偏移（种子 20260914，排除 0）。±7 天出现真值挤出的档位不得推荐。",
|
||||
"",
|
||||
])
|
||||
for span in JITTER_SPANS:
|
||||
lines.append(f"### ±{span} 天")
|
||||
lines.append("")
|
||||
table = []
|
||||
for radius in payload["radii"]:
|
||||
for gate in GATES:
|
||||
summary = (((m2.get(str(span)) or {}).get(str(radius)) or {}).get(gate) or {})
|
||||
table.append([
|
||||
f"±{radius} {gate}",
|
||||
summary.get("top1"),
|
||||
summary.get("coverage"),
|
||||
summary.get("squeezed"),
|
||||
summary.get("width_median"),
|
||||
])
|
||||
lines.append(md_table(["格子", "头名命中", "区间覆盖", "挤出", "宽度中位"], table))
|
||||
lines.append("")
|
||||
v3_jitter = payload.get("v3_jitter7") or {}
|
||||
if v3_jitter:
|
||||
lines.append("V3 在 ±7 天偏移下：")
|
||||
lines.append("")
|
||||
table = []
|
||||
for radius in payload["radii"]:
|
||||
summary = v3_jitter.get(str(radius)) or {}
|
||||
table.append([
|
||||
f"±{radius}",
|
||||
summary.get("top1"),
|
||||
summary.get("coverage"),
|
||||
summary.get("squeezed"),
|
||||
])
|
||||
lines.append(md_table(["半径", "头名命中", "区间覆盖", "挤出"], table))
|
||||
lines.append("")
|
||||
lines.extend([
|
||||
"## M3 · 六题之后还能再出几道",
|
||||
"",
|
||||
"数字见 M1 表的「六题后再出」与「有分辨力」。若新增 ≥8 道而宽度只收 1–2 分钟，判定里记为问答成本高于收益。",
|
||||
"",
|
||||
md_table(
|
||||
["半径", "档", "再出均", "有分辨力均", "宽度中位"],
|
||||
m3.get("rows") or [],
|
||||
),
|
||||
"",
|
||||
"## 方法",
|
||||
"",
|
||||
"1. 每例先算一次静态盘（与事件无关），再按精度处理 / 偏移 / 分盘政策套 `build_event_contribution_matrix`。",
|
||||
"2. 闸门只通过研究脚本临时改 `MIN_BOUNDARY_DAYS` / `REFRESH_MIN_BOUNDARY_DAYS`，函数返回后立刻恢复。",
|
||||
"3. 参与该题的证据精度取该例事件包里最细的一档（有日用日，否则月，否则年）。",
|
||||
"4. 六题回放用真值簇最优 yes/no（上界，不是真人会怎么答）。交付口径与簇宽度研究相同：lead 8 未淘汰并集。",
|
||||
"5. 刷新出题走生产 `refresh_probes=true`（含已落地的 R3/R4），只换边界天数闸。",
|
||||
"",
|
||||
f"错误 {len(payload.get('errors') or [])} 例。{payload.get('case_count', 0)} 例跑完。",
|
||||
"",
|
||||
])
|
||||
if payload.get("errors"):
|
||||
lines.append("失败例子：")
|
||||
for row in payload["errors"]:
|
||||
lines.append(f"- `{row.get('case_id')}` {row.get('error')}")
|
||||
lines.append("")
|
||||
REPORT_MD.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
slim = dict(payload)
|
||||
slim.pop("case_rows", None)
|
||||
REPORT_JSON.write_text(json.dumps(slim, ensure_ascii=True, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
CHECKPOINT = ROOT / "scratch" / "precision_gate_checkpoint.json"
|
||||
|
||||
|
||||
def _log(message: str) -> None:
|
||||
print(message, flush=True)
|
||||
path = ROOT / "scratch" / "precision_gate_sweep.log"
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("a", encoding="utf-8") as handle:
|
||||
handle.write(message + "\n")
|
||||
|
||||
|
||||
def _empty_state(radii: Sequence[int]) -> dict[str, Any]:
|
||||
return {
|
||||
"done": [],
|
||||
"m0_rows": {str(radius): {name: [] for name in TREATMENTS} for radius in radii},
|
||||
"m1_rows": {str(radius): {name: [] for name in GATES} for radius in radii},
|
||||
"m1b_rows": {str(radius): {name: [] for name in VARGA_NAMES} for radius in radii},
|
||||
"combo_rows": {str(radius): {} for radius in radii},
|
||||
"m2_rows": {
|
||||
str(span): {str(radius): {name: [] for name in GATES} for radius in radii}
|
||||
for span in JITTER_SPANS
|
||||
},
|
||||
"v3_jitter_rows": {str(radius): [] for radius in radii},
|
||||
"ablations": [],
|
||||
"errors": [],
|
||||
}
|
||||
|
||||
|
||||
def _save_checkpoint(state: dict[str, Any]) -> None:
|
||||
CHECKPOINT.parent.mkdir(parents=True, exist_ok=True)
|
||||
CHECKPOINT.write_text(json.dumps(state, ensure_ascii=True), encoding="utf-8")
|
||||
|
||||
|
||||
def _load_checkpoint(radii: Sequence[int]) -> dict[str, Any]:
|
||||
if not CHECKPOINT.exists():
|
||||
return _empty_state(radii)
|
||||
payload = json.loads(CHECKPOINT.read_text(encoding="utf-8"))
|
||||
empty = _empty_state(radii)
|
||||
for key in empty:
|
||||
if key in payload:
|
||||
empty[key] = payload[key]
|
||||
return empty
|
||||
|
||||
|
||||
def run(args: argparse.Namespace) -> dict[str, Any]:
|
||||
holdout = json.loads(HOLDOUT.read_text(encoding="utf-8"))
|
||||
cases = list(holdout["cases"])
|
||||
if args.limit:
|
||||
cases = cases[: args.limit]
|
||||
radii = tuple(int(item) for item in args.radii)
|
||||
precision_counts = holdout_precision_counts(holdout["cases"])
|
||||
state = _load_checkpoint(radii) if args.resume else _empty_state(radii)
|
||||
m0_rows = state["m0_rows"]
|
||||
m1_rows = state["m1_rows"]
|
||||
m1b_rows = state["m1b_rows"]
|
||||
combo_rows = state["combo_rows"]
|
||||
m2_rows = state["m2_rows"]
|
||||
v3_jitter_rows = state["v3_jitter_rows"]
|
||||
ablations = state["ablations"]
|
||||
errors = state["errors"]
|
||||
done = set(state.get("done") or [])
|
||||
assert event_probes.MIN_BOUNDARY_DAYS == 45
|
||||
assert event_probes.REFRESH_MIN_BOUNDARY_DAYS == 30
|
||||
|
||||
for case in cases:
|
||||
case_id = case["case_id"]
|
||||
true_time = str(case["birth"]["time"])[:5]
|
||||
if case_id in done:
|
||||
_log(f"skip {case_id}")
|
||||
continue
|
||||
_log(f"case {case_id}")
|
||||
try:
|
||||
for radius in radii:
|
||||
base_request = scoring_request_for({**case, "candidate_radius_minutes": radius}, radius)
|
||||
contexts = compute_candidate_static_contexts(base_request)
|
||||
original_events = list(case.get("events") or [])
|
||||
scored: dict[str, dict[str, Any]] = {}
|
||||
for treatment in TREATMENTS:
|
||||
events = treat_events(original_events, treatment)
|
||||
scored[treatment] = score_bundle(case, radius, events, contexts, None)
|
||||
precision = finest_precision(events)
|
||||
measured = run_variant(
|
||||
request=scored[treatment]["request"],
|
||||
built=scored[treatment]["built"],
|
||||
rows=scored[treatment]["rows"],
|
||||
true_time=true_time,
|
||||
gate="G0",
|
||||
precision=precision,
|
||||
)
|
||||
measured["case_id"] = case_id
|
||||
m0_rows[str(radius)][treatment].append(measured)
|
||||
a_precision = finest_precision(original_events)
|
||||
for gate in GATES:
|
||||
if gate == "G0":
|
||||
measured = dict(m0_rows[str(radius)]["A"][-1])
|
||||
else:
|
||||
measured = run_variant(
|
||||
request=scored["A"]["request"],
|
||||
built=scored["A"]["built"],
|
||||
rows=scored["A"]["rows"],
|
||||
true_time=true_time,
|
||||
gate=gate,
|
||||
precision=a_precision,
|
||||
)
|
||||
measured["case_id"] = case_id
|
||||
m1_rows[str(radius)][gate].append(measured)
|
||||
scores_a = {
|
||||
str(row["time"])[:5]: float(row.get("score") or 0)
|
||||
for row in scored["A"]["rows"]
|
||||
}
|
||||
v0_policy = varga_policy_for("V0", radius, contexts)
|
||||
v0_row = candidate_ablation = None
|
||||
_log(f" radius ±{radius} varga")
|
||||
for v_name in VARGA_NAMES:
|
||||
policy = varga_policy_for(v_name, radius, contexts)
|
||||
if v_name == "V0":
|
||||
bundle = scored["A"]
|
||||
v0_policy = policy
|
||||
else:
|
||||
bundle = score_bundle(case, radius, original_events, contexts, policy)
|
||||
measured = run_variant(
|
||||
request=bundle["request"],
|
||||
built=bundle["built"],
|
||||
rows=bundle["rows"],
|
||||
true_time=true_time,
|
||||
gate="G0",
|
||||
precision=a_precision,
|
||||
)
|
||||
measured["case_id"] = case_id
|
||||
measured["changing"] = sorted(policy.changing)
|
||||
m1b_rows[str(radius)][v_name].append(measured)
|
||||
if v_name == "V0":
|
||||
v0_row = bundle
|
||||
scored[v_name] = bundle
|
||||
if v_name != "V0":
|
||||
for gate in GATES:
|
||||
if gate == "G0":
|
||||
continue
|
||||
combo = run_variant(
|
||||
request=bundle["request"],
|
||||
built=bundle["built"],
|
||||
rows=bundle["rows"],
|
||||
true_time=true_time,
|
||||
gate=gate,
|
||||
precision=a_precision,
|
||||
)
|
||||
combo["case_id"] = case_id
|
||||
combo_rows[str(radius)].setdefault(f"{gate}+{v_name}", []).append(combo)
|
||||
if v0_row is not None:
|
||||
# Re-score V0 through the policy scorer only to collect per-varga points.
|
||||
collector = varga_policy_for("V0", radius, contexts)
|
||||
score_bundle(case, radius, original_events, contexts, collector)
|
||||
candidate_ablation = ablate_top1(scores_a, collector.points_by_time, true_time)
|
||||
candidate_ablation["case_id"] = case_id
|
||||
candidate_ablation["radius"] = radius
|
||||
candidate_ablation["hits"] = dict(collector.hits)
|
||||
ablations.append(candidate_ablation)
|
||||
if not args.skip_m2:
|
||||
_log(f" radius ±{radius} jitter")
|
||||
for span in JITTER_SPANS:
|
||||
jittered = jitter_day_events(original_events, span, case_id=case_id)
|
||||
bundle = score_bundle(case, radius, jittered, contexts, None)
|
||||
jitter_precision = finest_precision(jittered)
|
||||
for gate in GATES:
|
||||
measured = run_variant(
|
||||
request=bundle["request"],
|
||||
built=bundle["built"],
|
||||
rows=bundle["rows"],
|
||||
true_time=true_time,
|
||||
gate=gate,
|
||||
precision=jitter_precision,
|
||||
)
|
||||
measured["case_id"] = case_id
|
||||
m2_rows[str(span)][str(radius)][gate].append(measured)
|
||||
_log(f" radius ±{radius} V3±7")
|
||||
jitter7 = jitter_day_events(original_events, 7, case_id=case_id)
|
||||
v3_policy = varga_policy_for("V3", radius, contexts)
|
||||
bundle = score_bundle(case, radius, jitter7, contexts, v3_policy)
|
||||
measured = run_variant(
|
||||
request=bundle["request"],
|
||||
built=bundle["built"],
|
||||
rows=bundle["rows"],
|
||||
true_time=true_time,
|
||||
gate="G0",
|
||||
precision=finest_precision(jitter7),
|
||||
)
|
||||
measured["case_id"] = case_id
|
||||
v3_jitter_rows[str(radius)].append(measured)
|
||||
_log(
|
||||
f" radius ±{radius} A_probes={m1_rows[str(radius)]['G0'][-1]['probes']} "
|
||||
f"G3_probes={m1_rows[str(radius)]['G3'][-1]['probes']}",
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
errors.append({
|
||||
"case_id": case_id,
|
||||
"error": f"{type(exc).__name__}: {exc}",
|
||||
"trace": traceback.format_exc(),
|
||||
})
|
||||
_log(f" FAIL {case_id}: {exc}")
|
||||
done.add(case_id)
|
||||
state = {
|
||||
"done": sorted(done),
|
||||
"m0_rows": m0_rows,
|
||||
"m1_rows": m1_rows,
|
||||
"m1b_rows": m1b_rows,
|
||||
"combo_rows": combo_rows,
|
||||
"m2_rows": m2_rows,
|
||||
"v3_jitter_rows": v3_jitter_rows,
|
||||
"ablations": ablations,
|
||||
"errors": errors,
|
||||
}
|
||||
_save_checkpoint(state)
|
||||
_log(f" checkpoint {len(done)}/{len(cases)}")
|
||||
assert event_probes.MIN_BOUNDARY_DAYS == 45
|
||||
assert event_probes.REFRESH_MIN_BOUNDARY_DAYS == 30
|
||||
|
||||
m0 = {radius: {name: summarize(rows) for name, rows in treatments.items()} for radius, treatments in m0_rows.items()}
|
||||
m1 = {radius: {name: summarize(rows) for name, rows in gates.items()} for radius, gates in m1_rows.items()}
|
||||
m1b = {radius: {name: summarize(rows) for name, rows in variants.items()} for radius, variants in m1b_rows.items()}
|
||||
combos = {
|
||||
radius: {name: summarize(rows) for name, rows in variants.items()}
|
||||
for radius, variants in combo_rows.items()
|
||||
}
|
||||
m2 = {
|
||||
span: {
|
||||
radius: {gate: summarize(rows) for gate, rows in gates.items()}
|
||||
for radius, gates in radii_map.items()
|
||||
}
|
||||
for span, radii_map in m2_rows.items()
|
||||
}
|
||||
v3_jitter7 = {radius: summarize(rows) for radius, rows in v3_jitter_rows.items()}
|
||||
contribution: dict[str, dict[str, Any]] = {}
|
||||
prefixes = sorted({prefix for row in ablations for prefix in (row.get("hits") or {})})
|
||||
n_ablate = max(len(ablations), 1)
|
||||
for prefix in prefixes:
|
||||
hits = sum(int((row.get("hits") or {}).get(prefix) or 0) for row in ablations)
|
||||
flips = sum(int((row.get("flips") or {}).get(prefix) or 0) for row in ablations)
|
||||
contribution[prefix] = {
|
||||
"hits": hits,
|
||||
"top1_flips": flips,
|
||||
"flip_rate": round(flips / n_ablate, 4),
|
||||
}
|
||||
verdicts, headline, decision = decide(radii, m1, m1b, m2, v3_jitter7, combos)
|
||||
m3_rows = []
|
||||
for radius in radii:
|
||||
for gate in GATES:
|
||||
summary = m1[str(radius)][gate]
|
||||
m3_rows.append([
|
||||
f"±{radius}", gate,
|
||||
summary.get("refresh_mean"),
|
||||
summary.get("discriminative_mean"),
|
||||
summary.get("width_median"),
|
||||
])
|
||||
payload = {
|
||||
"generated_at": TODAY.isoformat(),
|
||||
"ayanamsa": AYANAMSA,
|
||||
"node_mode": NODE_MODE,
|
||||
"holdout": str(HOLDOUT.relative_to(ROOT)).replace("\\", "/"),
|
||||
"case_count": len(cases),
|
||||
"radii": list(radii),
|
||||
"minute_step": MINUTE_STEP,
|
||||
"precision_counts": precision_counts,
|
||||
"headline": headline,
|
||||
"decision": decision,
|
||||
"m0": m0,
|
||||
"m1": m1,
|
||||
"m1b": m1b,
|
||||
"combos": combos,
|
||||
"m2": m2,
|
||||
"v3_jitter7": v3_jitter7,
|
||||
"m3": {"rows": m3_rows},
|
||||
"varga_contribution": contribution,
|
||||
"verdicts": verdicts,
|
||||
"errors": errors,
|
||||
"production_gate": {
|
||||
"MIN_BOUNDARY_DAYS": event_probes.MIN_BOUNDARY_DAYS,
|
||||
"REFRESH_MIN_BOUNDARY_DAYS": event_probes.REFRESH_MIN_BOUNDARY_DAYS,
|
||||
},
|
||||
}
|
||||
write_report(payload)
|
||||
return payload
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--limit", type=int, default=0)
|
||||
parser.add_argument("--radii", nargs="+", default=[str(item) for item in RADII])
|
||||
parser.add_argument("--skip-m2", action="store_true")
|
||||
parser.add_argument("--resume", action="store_true")
|
||||
parser.add_argument("--from-json", action="store_true")
|
||||
args = parser.parse_args()
|
||||
if args.from_json:
|
||||
payload = json.loads(REPORT_JSON.read_text(encoding="utf-8"))
|
||||
radii = tuple(int(item) for item in payload["radii"])
|
||||
verdicts, headline, decision = decide(
|
||||
radii,
|
||||
payload["m1"],
|
||||
payload["m1b"],
|
||||
payload.get("m2") or {},
|
||||
payload.get("v3_jitter7") or {},
|
||||
payload.get("combos") or {},
|
||||
)
|
||||
payload["verdicts"] = verdicts
|
||||
payload["headline"] = headline
|
||||
payload["decision"] = decision
|
||||
write_report(payload)
|
||||
else:
|
||||
payload = run(args)
|
||||
print(
|
||||
f"wrote {REPORT_MD} cases={payload['case_count']} errors={len(payload['errors'])} "
|
||||
f"decision={payload['decision']}",
|
||||
flush=True,
|
||||
)
|
||||
return 0 if not payload["errors"] else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,142 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.rectification.event_probes import MIN_BOUNDARY_DAYS, REFRESH_MIN_BOUNDARY_DAYS
|
||||
import scripts.rectification.event_probes as event_probes
|
||||
from scripts.research.precision_gate_lib import (
|
||||
GATES,
|
||||
close_same_year_windows,
|
||||
count_precision,
|
||||
finest_precision,
|
||||
gate_verdict,
|
||||
jitter_day_events,
|
||||
patched_boundary_gate,
|
||||
production_defaults_intact,
|
||||
threshold_for,
|
||||
treat_events,
|
||||
varga_factor,
|
||||
varga_minutes,
|
||||
window_minutes_for_radius,
|
||||
)
|
||||
|
||||
HOLDOUT = Path("references/real_case_calibration/minute_rectification_holdout_v4.json")
|
||||
|
||||
|
||||
def _events() -> list[dict]:
|
||||
return [
|
||||
{"id": "day-1", "domain": "career", "date": "2011-08-24", "precision": "day"},
|
||||
{"id": "year-1", "domain": "education", "date": "2003", "precision": "year"},
|
||||
]
|
||||
|
||||
|
||||
def test_holdout_v4_has_the_stated_precision_mix() -> None:
|
||||
payload = json.loads(HOLDOUT.read_text(encoding="utf-8"))
|
||||
tallies = count_precision([
|
||||
event for case in payload["cases"] for event in case["events"]
|
||||
])
|
||||
assert tallies["day"] == 59
|
||||
assert tallies["year"] == 84
|
||||
assert tallies["month"] == 0
|
||||
assert len(payload["cases"]) == 20
|
||||
|
||||
|
||||
def test_treatment_b_downgrades_day_only() -> None:
|
||||
treated = treat_events(_events(), "B")
|
||||
assert treated[0]["precision"] == "month"
|
||||
assert treated[1]["precision"] == "year"
|
||||
|
||||
|
||||
def test_treatment_c_downgrades_everything_to_year() -> None:
|
||||
treated = treat_events(_events(), "C")
|
||||
assert {item["precision"] for item in treated} == {"year"}
|
||||
|
||||
|
||||
def test_finest_precision_prefers_day() -> None:
|
||||
assert finest_precision(_events()) == "day"
|
||||
assert finest_precision(treat_events(_events(), "B")) == "month"
|
||||
assert finest_precision(treat_events(_events(), "C")) == "year"
|
||||
|
||||
|
||||
def test_g0_refresh_stays_at_production_30() -> None:
|
||||
assert threshold_for("G0", "day", refresh=False) == 45
|
||||
assert threshold_for("G0", "day", refresh=True) == 30
|
||||
assert threshold_for("G1", "day", refresh=True) == 10
|
||||
assert threshold_for("G3", "day", refresh=False) == 5
|
||||
assert set(GATES) == {"G0", "G1", "G2", "G3", "G4"}
|
||||
|
||||
|
||||
def test_jitter_stays_inside_the_span_and_is_reproducible() -> None:
|
||||
first = jitter_day_events(_events(), 7, case_id="demo")
|
||||
second = jitter_day_events(_events(), 7, case_id="demo")
|
||||
assert first[0]["date"] == second[0]["date"]
|
||||
original = date.fromisoformat("2011-08-24")
|
||||
shifted = date.fromisoformat(first[0]["date"])
|
||||
delta = abs((shifted - original).days)
|
||||
assert 1 <= delta <= 7
|
||||
assert first[1]["date"] == "2003"
|
||||
|
||||
|
||||
def test_varga_factor_downweights_slow_charts_in_a_20_minute_window() -> None:
|
||||
assert window_minutes_for_radius(10) == 20
|
||||
assert varga_minutes("D30") == 4.0
|
||||
assert varga_minutes("D2") == 60.0
|
||||
assert varga_factor("D30", 20) == 1.0
|
||||
assert abs(varga_factor("D2", 20) - 20 / 60) < 1e-9
|
||||
assert varga_minutes("D4") == 7.5
|
||||
|
||||
|
||||
def test_forty_day_same_year_window_needs_the_lower_gate() -> None:
|
||||
left = date(2001, 3, 1)
|
||||
right = date(2001, 4, 10)
|
||||
assert close_same_year_windows(left, right, 45) == []
|
||||
kept = close_same_year_windows(left, right, 30)
|
||||
assert [item.month for item in kept] == [3, 4]
|
||||
|
||||
|
||||
def test_patched_gate_restores_production_defaults() -> None:
|
||||
assert event_probes.MIN_BOUNDARY_DAYS == 45
|
||||
with patched_boundary_gate(initial=10, refresh=7):
|
||||
assert event_probes.MIN_BOUNDARY_DAYS == 10
|
||||
assert event_probes.REFRESH_MIN_BOUNDARY_DAYS == 7
|
||||
assert event_probes.MIN_BOUNDARY_DAYS == 45
|
||||
assert event_probes.REFRESH_MIN_BOUNDARY_DAYS == 30
|
||||
assert production_defaults_intact()
|
||||
assert MIN_BOUNDARY_DAYS == 45
|
||||
assert REFRESH_MIN_BOUNDARY_DAYS == 30
|
||||
|
||||
|
||||
def test_verdict_blocks_a_plus_minus_seven_squeeze() -> None:
|
||||
baseline = {
|
||||
"n": 20, "top1": 0.8, "coverage": 1.0, "width_median": 15,
|
||||
"squeezed": 0, "probes_mean": 4, "refresh_mean": 1, "discriminative_mean": 1,
|
||||
}
|
||||
narrower = {**baseline, "width_median": 9, "refresh_mean": 3, "discriminative_mean": 2}
|
||||
squeezed = {**narrower, "coverage": 0.95, "squeezed": 1}
|
||||
assert gate_verdict(baseline, narrower, jitter7_squeezed=0) == "benefit"
|
||||
assert gate_verdict(baseline, narrower, jitter7_squeezed=1) == "no_benefit"
|
||||
assert gate_verdict(baseline, squeezed, jitter7_squeezed=0) == "no_benefit"
|
||||
|
||||
|
||||
def test_eight_extra_questions_for_two_minutes_is_not_worth_it() -> None:
|
||||
baseline = {
|
||||
"n": 20, "top1": 0.8, "coverage": 1.0, "width_median": 15,
|
||||
"squeezed": 0, "probes_mean": 4, "refresh_mean": 1, "discriminative_mean": 1,
|
||||
}
|
||||
costly = {
|
||||
**baseline, "width_median": 13, "refresh_mean": 10, "discriminative_mean": 8,
|
||||
}
|
||||
assert gate_verdict(baseline, costly, jitter7_squeezed=0) == "no_benefit"
|
||||
|
||||
|
||||
def test_more_probes_with_a_wider_range_is_not_a_benefit() -> None:
|
||||
baseline = {
|
||||
"n": 20, "top1": 0.55, "coverage": 1.0, "width_median": 33,
|
||||
"squeezed": 0, "probes_mean": 7.1, "refresh_mean": 5.45, "discriminative_mean": 5.45,
|
||||
}
|
||||
wider = {
|
||||
**baseline, "width_median": 37, "refresh_mean": 5.5, "discriminative_mean": 5.5,
|
||||
}
|
||||
assert gate_verdict(baseline, wider, jitter7_squeezed=0) == "no_benefit"
|
||||
Reference in New Issue
Block a user