Compare commits

...
Author SHA1 Message Date
jesse-ux 2b7b45657b test(rectification): prove memoization in-process; tolerate golden float drift (BUG-733)
Independent Staging Quality Gate / validate (push) Canceled after 5m0s
Independent Staging Quality Gate / publish (push) Canceled after 0s
Replace whole-structure golden == with same-process cached vs None fallback equality, plus discrete-strict / measured-tolerance golden comparison. Do not rebuild the golden JSON.
2026-09-16 07:56:56 +08:00
3 changed files with 243 additions and 2 deletions
+16
View File
@@ -11396,3 +11396,19 @@
- 相关记录:BUG-464 - 相关记录:BUG-464
- 复发自:无 - 复发自:无
- 修复版本:待发布 - 修复版本:待发布
## BUG-733 | 校正记忆化 golden 对全精度浮点做整体 `==`,跨机门禁靠运气绿
- 状态:resolved
- 首次发现:2026-09-15
- 最近更新:2026-09-16
- 影响面:`tests/test_rectification_engine_memoization.py``tests/golden/rectification_engine_memoization_v1.json`(文件未改)
- 用户现象:用户看不见。门禁 glob `tests/test_rectification_*.py``CORE_PYTEST_TARGETS` 里,换一台机器或基础镜像会因 `score` / `margin_percent` 尾数差 1.0e-41.1e-3 而红,CI 绿只是碰巧和生成 golden 的那台舍入一致。
- 触发条件:`test_score_candidates_matches_baseline_golden` 把 live payload 与 golden 做整体 `==`
- 根因:golden 存了全精度浮点并整体相等。这是 BUG-712 的同一形状。BUG-712 的防复发只落在 `ephemeris_events` 那一处(量化到 6 位再比),没有仓库级守卫,BUG-721 新写的 rectification golden 重蹈覆辙。
- 修复:只改测试,不动 `scripts/`。主证据改成同进程差分:同一批 static contextsA 带四层缓存、B 把 `ashtakavarga_result` / `shadbala_result` / `vimshottari_timeline` / `narayana_periods``None``compute_event_candidate_rows` 输出严格相等。golden 改成分档比较:离散字段严格相等;浮点用实测最大漂移 1.1e-3 推出的容差(绝对 2e-3 与相对 5e-4 取更宽者)。禁止调用 `write_golden()` 更新那份 JSON。
- 验证:同进程 A/B 逐字相等,且 B 的 `calc_shadbala` 次数显著高于 A(本机 6 vs 0)。反向:把某一层缓存换成全零 shadbala 对象后不再相等。golden 分档比较通过;把 golden 浮点改 1e-2 必须红;把离散字段 `time` 改掉必须红。`pytest tests/test_rectification_engine_memoization.py``pytest tests/test_rectification_*.py` 0 failed。golden JSON `git diff` 无改动。
- 防复发:任何 golden 比较都不得对浮点做整体 `==`;浮点必须量化或带容差,容差数值要有实测依据并写在注释里;能在同进程内做差分证明的命题,不得用跨机 golden 代替。
- 相关记录:BUG-712、BUG-721
- 复发自:BUG-712
- 修复版本:待发布
@@ -0,0 +1,64 @@
# PROGRESS · 等价 golden 跨机不稳(2026-09-15 / 执行 2026-09-16
- 执行分支:`codex/rectification-engine-memoization-fix-20260915`
- 工作树:`.worktrees/rectification-engine-memoization-fix-20260915`
- 任务书:`docs/tasks/TASK-rectification-engine-memoization-fix-20260915.md`
- 开工 `HEAD` = `origin/staging` = `51a65d92`(用户核对时是 `7227b1ed`,其后合入了 BUG-732
- BUG`BUG-733`(复发自 BUG-712;落库时最大号 732
## 开工前置
| 项 | 结果 |
| --- | --- |
| `git status -sb` 第一行 | `## codex/rectification-engine-memoization-fix-20260915...origin/staging` |
| `HEAD` | `51a65d92` |
| BUG_HISTORY | 最大号 **BUG-732**。本单预占 **BUG-733**。读完 BUG-712 / BUG-721 |
| 本机 Python | 主仓 `.venv` 是 25 字节指针文件。本 worktree 用 Anaconda 3.11.7 `--system-site-packages` 建真实 venv,复用 `swisseph 20230604`,未升级依赖 |
| 先复现 golden | `test_score_candidates_matches_baseline_golden` **本机绿**。live 与 golden 逐字相同:`score` 8.6274 / 8.6227 / 8.1701`margin_percent` 5.3006。验收机上的 1.1e-3 漂移在这台 Windows / Anaconda 3.11.7 上复现不出来——golden 就是同类机器生成的。仍按任务书改比较方式,不调用 `write_golden()` |
未改 `docs/tasks/README.md` 状态列。未推 staging。未改任何 `scripts/` 实现。未改 `tests/golden/rectification_engine_memoization_v1.json`
## 做了什么
| 条 | 做法 |
| --- | --- |
| 5.1 | 新增 `test_cached_static_context_matches_uncached_fallback`:同一请求、同一进程,A=带四层缓存的 static contextsB=四键置 `None``compute_event_candidate_rows` 输出 `==`。调用计数:A 的 `calc_shadbala` = 0,B = 6(3 候选 × 2 事件)。反向:`test_poisoned_static_cache_diverges_from_live_rows` 把第一候选的 `shadbala_result` 换成全零对象,分数从 12.5125 变成 12.6125,断言不再相等 |
| 5.2 | `test_score_candidates_matches_baseline_golden` 改成分档比较。整数 / 布尔 / 字符串 / 列表身份严格相等;浮点容差 `max(2e-3, 5e-4 * \|expected\|)`,注释写明来自实测最大漂移 1.1e-3。反向:`score += 1e-2` 红;`time = "99:99"` 红 |
| 5.3 | 未调用 `write_golden()``git diff` 里该 JSON 无改动 |
| 5.4 | 六份检查结论见下表。同类问题只记录不修 |
| 5.5 | `docs/BUG_HISTORY.md` 新增 BUG-733,复发自 BUG-712。证据是 5.1 同进程差分 + 两次反向验证,不是「现在绿了」 |
## 5.4 `tests/golden/` 检查
| 文件 | 全精度浮点整体 `==` | 在哪个测试里比 | 怎么比 |
| --- | --- | --- | --- |
| `consultation_contract_keypaths_v1.json` | 无 | `tests/test_consultation_contract_golden.py` | 只存 keypath 与 JSON 类型(`kind=keypath_types_only`)。值不入库。live 是 keypath 超集 + 类型相同 |
| `ephemeris_events_raman_20260915_90d.json` | 无(BUG-712 已改) | `tests/test_ephemeris_events.py` | `longitude` / `speed_longitude` 量化到 6 位再 `==``kind` / `date` / `body` / 星座字段严格相等。文件里仍存全精度,但测试不再整体 `==` |
| `golden_cases.json` | 无 | `tests/test_cli_smoke.py::test_full_reading_golden_cases_cover_user_ready_output``tests/run_golden_cases.py` | 路径非空、模块计数、整数不变量 `ashtakavarga_sav_total == 337`、日期覆盖。不是把整份引擎浮点输出整体 `==` |
| `qizheng_stem_branch_19900409.json` | 无(比较已量化) | `tests/test_qizheng_chart_engine.py` | 离散字段(宫、宿、ketuMode)严格相等;`siderealLon` / `mansionDegree``round(..., 2)`。**文件本身仍存全精度浮点**(如 `177.94225865978694`)。若有人新写 `assert result == golden` 会变成 BUG-712 同类。本单只记录不修 |
| `rectification_engine_memoization_v1.json` | **本单之前有** | `tests/test_rectification_engine_memoization.py::test_score_candidates_matches_baseline_golden` | 改前:`candidate_scores``decision_receipt` 整体 `==`。改后:离散严格、浮点容差。文件未重建 |
| `upstream_sync2/` | 无整体浮点 `==` | 见分项 | `relationship_einstein.json``test_upstream_sync2_goldens.py` 只比键与类型。`consultation_marriage_evidence_snapshot.json`shape。`consultation_marriage_1990_fictional.json``test_relationship_event_class_evidence.py` 比离散 Dasha 字符串。`pl9_time_system_table.baseline.md``test_full_reading_conditional_dashas.py` 比对 markdown 表行文本(前 5 行 `==`),不是 JSON 浮点。`pl9_time_system_table.md``pl9_export_timing.json`(含 `seconds: 8.647`**没有任何测试读取** |
同类观察(不修):qizheng golden 文件仍存全精度;ephemeris golden 文件仍存全精度(测试已量化);`upstream_sync2/pl9_export_timing.json` 是无引用的浮点秒数。
## 测试
| 套件 | 结果 |
| --- | --- |
| `tests/test_rectification_engine_memoization.py` | **14 passed**(原 10 + 新增 4 |
| `tests/test_rectification_*.py` | **186 passed, 0 failed**14.42s |
命令使用 `.\.venv\Scripts\python.exe`(任务书写的 `.venv/bin/python` 在 Windows 不存在)。
未改任何既有断言的期望值。新增 4 条:同进程差分、缓存毒化反向、浮点 1e-2 反向、离散字段反向。原 golden 断言从整体 `==` 改成分档,不是放宽离散字段。未跑 `run_quality_gate.py --profile quick`:该 profile 在 pytest 之后会跑 `npm test`,本机 Windows 缺 `tsx` PATH(与 BUG-721 进度记录同一缺口),本单未改前端。
## CHANGELOG
未改。纯测试比较方式,用户可见打分不变。
## 偏离
- 任务书写 `.venv/bin/python`,本机换成 `.\.venv\Scripts\python.exe`
- 本机未能复现验收机上的 golden 红灯(舍入与生成机一致)。仍改比较方式,并保留验收机测到的 1.1e-3 作为容差依据。
-`git push origin HEAD:staging`
- 5.4 按让步只检查并记录,未加仓库级自动扫描守卫。
+163 -2
View File
@@ -2,17 +2,25 @@
Golden payload in tests/golden/rectification_engine_memoization_v1.json was Golden payload in tests/golden/rectification_engine_memoization_v1.json was
produced from origin/staging @ a8d29d1b before any memoization landed. produced from origin/staging @ a8d29d1b before any memoization landed.
Do not compare that payload with a whole-structure ``==``. Cross-machine
libm / pyswisseph rounding already drifted ``margin_percent`` by 1.1e-3
(TASK-rectification-engine-memoization-fix-20260915). Equivalence of the
four cached layers is proven in-process instead.
""" """
from __future__ import annotations from __future__ import annotations
import inspect import inspect
import json import json
import math
from datetime import date, datetime from datetime import date, datetime
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from unittest.mock import patch from unittest.mock import patch
import pytest
from scripts.active_rectification_event_engine import ( from scripts.active_rectification_event_engine import (
_candidate_datetimes, _candidate_datetimes,
_controlled_transit_rules, _controlled_transit_rules,
@@ -35,6 +43,20 @@ GOLDEN_PATH = ROOT / "tests" / "golden" / "rectification_engine_memoization_v1.j
FROZEN_TODAY = date(2026, 9, 16) FROZEN_TODAY = date(2026, 9, 16)
TIMING_KEYS = frozenset({"column_compare_ms"}) TIMING_KEYS = frozenset({"column_compare_ms"})
SOURCE_COMMIT = "a8d29d1b6cc37ff865ddec6c8bccdf9aa889ee53" SOURCE_COMMIT = "a8d29d1b6cc37ff865ddec6c8bccdf9aa889ee53"
CACHE_LAYER_KEYS = (
"ashtakavarga_result",
"shadbala_result",
"vimshottari_timeline",
"narayana_periods",
)
# TASK-rectification-engine-memoization-fix-20260915 §2 measured max
# cross-machine drift of 1.1e-3 on decision_receipt.margin_percent
# (5.2995 vs golden 5.3006). Candidate scores, already rounded to 4
# decimals, drifted by 1 ulp (1.0e-4). Tolerance is the wider of
# abs=2e-3 and rel=5e-4 so 1.1e-3 still passes while a 1e-2 mutation
# (the reverse test below) fails.
GOLDEN_FLOAT_ABS = 2e-3
GOLDEN_FLOAT_REL = 5e-4
def public_score_request() -> dict[str, Any]: def public_score_request() -> dict[str, Any]:
@@ -117,11 +139,102 @@ def _count_calls(monkeypatch, owner: Any, name: str, *, from_engine: bool = Fals
return counter return counter
def _is_json_number(value: Any) -> bool:
return isinstance(value, (int, float)) and not isinstance(value, bool)
def _assert_tiered_equal(actual: Any, expected: Any, *, path: str) -> None:
if isinstance(expected, dict):
assert isinstance(actual, dict), path
assert set(actual) == set(expected), f"{path} keys {set(actual)!r} != {set(expected)!r}"
for key in expected:
_assert_tiered_equal(actual[key], expected[key], path=f"{path}.{key}")
return
if isinstance(expected, list):
assert isinstance(actual, list), path
assert len(actual) == len(expected), f"{path} length {len(actual)} != {len(expected)}"
for index, (left, right) in enumerate(zip(actual, expected)):
_assert_tiered_equal(left, right, path=f"{path}[{index}]")
return
if isinstance(expected, int) and isinstance(actual, int) and not isinstance(expected, bool) and not isinstance(actual, bool):
assert actual == expected, f"{path}: {actual!r} != {expected!r}"
return
if _is_json_number(expected) or _is_json_number(actual):
left = float(actual)
right = float(expected)
tolerance = max(GOLDEN_FLOAT_ABS, GOLDEN_FLOAT_REL * abs(right))
assert math.isclose(left, right, rel_tol=0.0, abs_tol=tolerance), (
f"{path}: {left!r} vs {right!r} exceeds {tolerance}"
)
return
assert actual == expected, f"{path}: {actual!r} != {expected!r}"
def _assert_memoization_payloads(actual: dict[str, Any], expected: dict[str, Any]) -> None:
_assert_tiered_equal(actual["candidate_scores"], expected["candidate_scores"], path="candidate_scores")
_assert_tiered_equal(actual["decision_receipt"], expected["decision_receipt"], path="decision_receipt")
def _event_engine_request() -> dict[str, Any]:
"""Dated events for compute_event_candidate_rows (not the v5 score payload)."""
return {
"birth_date": "1990-01-01",
"start_time": "12:00",
"end_time": "12:02",
"lat": 39.9,
"lon": 116.4,
"tz": 8,
"events": [
{
"id": "00000000-0000-4000-8000-000000000001",
"domain": "education",
"event_kind": "education_start",
"date": "2008-07-01",
"precision": "day",
"summary": "入学",
},
{
"id": "00000000-0000-4000-8000-000000000002",
"domain": "career",
"event_kind": "career_entry",
"date": "2012-06-15",
"precision": "day",
"summary": "入职",
},
],
}
def _contexts_with_layer_cache_cleared(contexts: list[dict[str, Any]]) -> list[dict[str, Any]]:
cleared: list[dict[str, Any]] = []
for context in contexts:
clone = dict(context)
for key in CACHE_LAYER_KEYS:
clone[key] = None
cleared.append(clone)
return cleared
def test_score_candidates_matches_baseline_golden() -> None: def test_score_candidates_matches_baseline_golden() -> None:
expected = json.loads(GOLDEN_PATH.read_text(encoding="utf-8")) expected = json.loads(GOLDEN_PATH.read_text(encoding="utf-8"))
actual = json.loads(json.dumps(_golden_payload(), ensure_ascii=True)) actual = json.loads(json.dumps(_golden_payload(), ensure_ascii=True))
assert actual["candidate_scores"] == expected["candidate_scores"] _assert_memoization_payloads(actual, expected)
assert actual["decision_receipt"] == expected["decision_receipt"]
def test_golden_float_shift_of_1e_minus_2_fails() -> None:
expected = json.loads(GOLDEN_PATH.read_text(encoding="utf-8"))
mutated = json.loads(json.dumps(expected))
mutated["candidate_scores"][0]["score"] += 1e-2
with pytest.raises(AssertionError):
_assert_memoization_payloads(expected, mutated)
def test_golden_discrete_field_mismatch_fails() -> None:
expected = json.loads(GOLDEN_PATH.read_text(encoding="utf-8"))
mutated = json.loads(json.dumps(expected))
mutated["candidate_scores"][0]["time"] = "99:99"
with pytest.raises(AssertionError):
_assert_memoization_payloads(expected, mutated)
def test_shadbala_verified_fields_match_with_and_without_birth_minute() -> None: def test_shadbala_verified_fields_match_with_and_without_birth_minute() -> None:
@@ -372,3 +485,51 @@ def test_compute_event_candidate_rows_reuses_static_context_without_mutating_it(
assert len(rows) == 2 assert len(rows) == 2
assert [frozenset(context) for context in contexts] == original_keys assert [frozenset(context) for context in contexts] == original_keys
assert all("_transit_chart_cache" not in context for context in contexts) assert all("_transit_chart_cache" not in context for context in contexts)
def test_cached_static_context_matches_uncached_fallback(monkeypatch) -> None:
request = _event_engine_request()
cached = [
build_candidate_static_context(request, candidate)
for candidate in _candidate_datetimes(request)
]
assert len(cached) >= 2
assert len(request["events"]) >= 2
for context in cached:
for key in CACHE_LAYER_KEYS:
assert context.get(key) is not None
uncached = _contexts_with_layer_cache_cleared(cached)
for context in uncached:
for key in CACHE_LAYER_KEYS:
assert context[key] is None
calls = _count_calls(monkeypatch, event_engine.shadbala, "calc_shadbala")
rows_cached = compute_event_candidate_rows(request, static_contexts=cached)
cached_calls = calls[0]
rows_uncached = compute_event_candidate_rows(request, static_contexts=uncached)
uncached_calls = calls[0] - cached_calls
assert rows_cached == rows_uncached
assert cached_calls == 0
assert uncached_calls > cached_calls
assert uncached_calls >= len(cached) * len(request["events"])
def test_poisoned_static_cache_diverges_from_live_rows() -> None:
request = _event_engine_request()
cached = [
build_candidate_static_context(request, candidate)
for candidate in _candidate_datetimes(request)
]
rows_cached = compute_event_candidate_rows(request, static_contexts=cached)
poisoned = [dict(context) for context in cached]
poisoned[0]["shadbala_result"] = {
"planets": {
name: {
"sthana_bala": {"total": 0.0},
"drik_bala": 0.0,
"naisargika_bala": 0.0,
}
for name in ("Sun", "Moon", "Mars", "Mercury", "Jupiter", "Venus", "Saturn")
}
}
rows_poisoned = compute_event_candidate_rows(request, static_contexts=poisoned)
assert rows_poisoned != rows_cached