test(rectification): prove memoization in-process; tolerate golden float drift (BUG-733)
Independent Staging Quality Gate / validate (push) Canceled after 5m0s
Independent Staging Quality Gate / publish (push) Canceled after 0s

Replace whole-structure golden == with same-process cached vs None fallback equality, plus discrete-strict / measured-tolerance golden comparison. Do not rebuild the golden JSON.
This commit is contained in:
jesse-ux
2026-09-16 07:56:56 +08:00
parent 51a65d927f
commit 2b7b45657b
3 changed files with 243 additions and 2 deletions
+16
View File
@@ -11396,3 +11396,19 @@
- 相关记录:BUG-464
- 复发自:无
- 修复版本:待发布
## BUG-733 | 校正记忆化 golden 对全精度浮点做整体 `==`,跨机门禁靠运气绿
- 状态:resolved
- 首次发现:2026-09-15
- 最近更新:2026-09-16
- 影响面:`tests/test_rectification_engine_memoization.py``tests/golden/rectification_engine_memoization_v1.json`(文件未改)
- 用户现象:用户看不见。门禁 glob `tests/test_rectification_*.py``CORE_PYTEST_TARGETS` 里,换一台机器或基础镜像会因 `score` / `margin_percent` 尾数差 1.0e-41.1e-3 而红,CI 绿只是碰巧和生成 golden 的那台舍入一致。
- 触发条件:`test_score_candidates_matches_baseline_golden` 把 live payload 与 golden 做整体 `==`
- 根因:golden 存了全精度浮点并整体相等。这是 BUG-712 的同一形状。BUG-712 的防复发只落在 `ephemeris_events` 那一处(量化到 6 位再比),没有仓库级守卫,BUG-721 新写的 rectification golden 重蹈覆辙。
- 修复:只改测试,不动 `scripts/`。主证据改成同进程差分:同一批 static contextsA 带四层缓存、B 把 `ashtakavarga_result` / `shadbala_result` / `vimshottari_timeline` / `narayana_periods``None``compute_event_candidate_rows` 输出严格相等。golden 改成分档比较:离散字段严格相等;浮点用实测最大漂移 1.1e-3 推出的容差(绝对 2e-3 与相对 5e-4 取更宽者)。禁止调用 `write_golden()` 更新那份 JSON。
- 验证:同进程 A/B 逐字相等,且 B 的 `calc_shadbala` 次数显著高于 A(本机 6 vs 0)。反向:把某一层缓存换成全零 shadbala 对象后不再相等。golden 分档比较通过;把 golden 浮点改 1e-2 必须红;把离散字段 `time` 改掉必须红。`pytest tests/test_rectification_engine_memoization.py``pytest tests/test_rectification_*.py` 0 failed。golden JSON `git diff` 无改动。
- 防复发:任何 golden 比较都不得对浮点做整体 `==`;浮点必须量化或带容差,容差数值要有实测依据并写在注释里;能在同进程内做差分证明的命题,不得用跨机 golden 代替。
- 相关记录:BUG-712、BUG-721
- 复发自:BUG-712
- 修复版本:待发布
@@ -0,0 +1,64 @@
# PROGRESS · 等价 golden 跨机不稳(2026-09-15 / 执行 2026-09-16
- 执行分支:`codex/rectification-engine-memoization-fix-20260915`
- 工作树:`.worktrees/rectification-engine-memoization-fix-20260915`
- 任务书:`docs/tasks/TASK-rectification-engine-memoization-fix-20260915.md`
- 开工 `HEAD` = `origin/staging` = `51a65d92`(用户核对时是 `7227b1ed`,其后合入了 BUG-732
- BUG`BUG-733`(复发自 BUG-712;落库时最大号 732
## 开工前置
| 项 | 结果 |
| --- | --- |
| `git status -sb` 第一行 | `## codex/rectification-engine-memoization-fix-20260915...origin/staging` |
| `HEAD` | `51a65d92` |
| BUG_HISTORY | 最大号 **BUG-732**。本单预占 **BUG-733**。读完 BUG-712 / BUG-721 |
| 本机 Python | 主仓 `.venv` 是 25 字节指针文件。本 worktree 用 Anaconda 3.11.7 `--system-site-packages` 建真实 venv,复用 `swisseph 20230604`,未升级依赖 |
| 先复现 golden | `test_score_candidates_matches_baseline_golden` **本机绿**。live 与 golden 逐字相同:`score` 8.6274 / 8.6227 / 8.1701`margin_percent` 5.3006。验收机上的 1.1e-3 漂移在这台 Windows / Anaconda 3.11.7 上复现不出来——golden 就是同类机器生成的。仍按任务书改比较方式,不调用 `write_golden()` |
未改 `docs/tasks/README.md` 状态列。未推 staging。未改任何 `scripts/` 实现。未改 `tests/golden/rectification_engine_memoization_v1.json`
## 做了什么
| 条 | 做法 |
| --- | --- |
| 5.1 | 新增 `test_cached_static_context_matches_uncached_fallback`:同一请求、同一进程,A=带四层缓存的 static contextsB=四键置 `None``compute_event_candidate_rows` 输出 `==`。调用计数:A 的 `calc_shadbala` = 0,B = 6(3 候选 × 2 事件)。反向:`test_poisoned_static_cache_diverges_from_live_rows` 把第一候选的 `shadbala_result` 换成全零对象,分数从 12.5125 变成 12.6125,断言不再相等 |
| 5.2 | `test_score_candidates_matches_baseline_golden` 改成分档比较。整数 / 布尔 / 字符串 / 列表身份严格相等;浮点容差 `max(2e-3, 5e-4 * \|expected\|)`,注释写明来自实测最大漂移 1.1e-3。反向:`score += 1e-2` 红;`time = "99:99"` 红 |
| 5.3 | 未调用 `write_golden()``git diff` 里该 JSON 无改动 |
| 5.4 | 六份检查结论见下表。同类问题只记录不修 |
| 5.5 | `docs/BUG_HISTORY.md` 新增 BUG-733,复发自 BUG-712。证据是 5.1 同进程差分 + 两次反向验证,不是「现在绿了」 |
## 5.4 `tests/golden/` 检查
| 文件 | 全精度浮点整体 `==` | 在哪个测试里比 | 怎么比 |
| --- | --- | --- | --- |
| `consultation_contract_keypaths_v1.json` | 无 | `tests/test_consultation_contract_golden.py` | 只存 keypath 与 JSON 类型(`kind=keypath_types_only`)。值不入库。live 是 keypath 超集 + 类型相同 |
| `ephemeris_events_raman_20260915_90d.json` | 无(BUG-712 已改) | `tests/test_ephemeris_events.py` | `longitude` / `speed_longitude` 量化到 6 位再 `==``kind` / `date` / `body` / 星座字段严格相等。文件里仍存全精度,但测试不再整体 `==` |
| `golden_cases.json` | 无 | `tests/test_cli_smoke.py::test_full_reading_golden_cases_cover_user_ready_output``tests/run_golden_cases.py` | 路径非空、模块计数、整数不变量 `ashtakavarga_sav_total == 337`、日期覆盖。不是把整份引擎浮点输出整体 `==` |
| `qizheng_stem_branch_19900409.json` | 无(比较已量化) | `tests/test_qizheng_chart_engine.py` | 离散字段(宫、宿、ketuMode)严格相等;`siderealLon` / `mansionDegree``round(..., 2)`。**文件本身仍存全精度浮点**(如 `177.94225865978694`)。若有人新写 `assert result == golden` 会变成 BUG-712 同类。本单只记录不修 |
| `rectification_engine_memoization_v1.json` | **本单之前有** | `tests/test_rectification_engine_memoization.py::test_score_candidates_matches_baseline_golden` | 改前:`candidate_scores``decision_receipt` 整体 `==`。改后:离散严格、浮点容差。文件未重建 |
| `upstream_sync2/` | 无整体浮点 `==` | 见分项 | `relationship_einstein.json``test_upstream_sync2_goldens.py` 只比键与类型。`consultation_marriage_evidence_snapshot.json`shape。`consultation_marriage_1990_fictional.json``test_relationship_event_class_evidence.py` 比离散 Dasha 字符串。`pl9_time_system_table.baseline.md``test_full_reading_conditional_dashas.py` 比对 markdown 表行文本(前 5 行 `==`),不是 JSON 浮点。`pl9_time_system_table.md``pl9_export_timing.json`(含 `seconds: 8.647`**没有任何测试读取** |
同类观察(不修):qizheng golden 文件仍存全精度;ephemeris golden 文件仍存全精度(测试已量化);`upstream_sync2/pl9_export_timing.json` 是无引用的浮点秒数。
## 测试
| 套件 | 结果 |
| --- | --- |
| `tests/test_rectification_engine_memoization.py` | **14 passed**(原 10 + 新增 4 |
| `tests/test_rectification_*.py` | **186 passed, 0 failed**14.42s |
命令使用 `.\.venv\Scripts\python.exe`(任务书写的 `.venv/bin/python` 在 Windows 不存在)。
未改任何既有断言的期望值。新增 4 条:同进程差分、缓存毒化反向、浮点 1e-2 反向、离散字段反向。原 golden 断言从整体 `==` 改成分档,不是放宽离散字段。未跑 `run_quality_gate.py --profile quick`:该 profile 在 pytest 之后会跑 `npm test`,本机 Windows 缺 `tsx` PATH(与 BUG-721 进度记录同一缺口),本单未改前端。
## CHANGELOG
未改。纯测试比较方式,用户可见打分不变。
## 偏离
- 任务书写 `.venv/bin/python`,本机换成 `.\.venv\Scripts\python.exe`
- 本机未能复现验收机上的 golden 红灯(舍入与生成机一致)。仍改比较方式,并保留验收机测到的 1.1e-3 作为容差依据。
-`git push origin HEAD:staging`
- 5.4 按让步只检查并记录,未加仓库级自动扫描守卫。
+163 -2
View File
@@ -2,17 +2,25 @@
Golden payload in tests/golden/rectification_engine_memoization_v1.json was
produced from origin/staging @ a8d29d1b before any memoization landed.
Do not compare that payload with a whole-structure ``==``. Cross-machine
libm / pyswisseph rounding already drifted ``margin_percent`` by 1.1e-3
(TASK-rectification-engine-memoization-fix-20260915). Equivalence of the
four cached layers is proven in-process instead.
"""
from __future__ import annotations
import inspect
import json
import math
from datetime import date, datetime
from pathlib import Path
from typing import Any
from unittest.mock import patch
import pytest
from scripts.active_rectification_event_engine import (
_candidate_datetimes,
_controlled_transit_rules,
@@ -35,6 +43,20 @@ GOLDEN_PATH = ROOT / "tests" / "golden" / "rectification_engine_memoization_v1.j
FROZEN_TODAY = date(2026, 9, 16)
TIMING_KEYS = frozenset({"column_compare_ms"})
SOURCE_COMMIT = "a8d29d1b6cc37ff865ddec6c8bccdf9aa889ee53"
CACHE_LAYER_KEYS = (
"ashtakavarga_result",
"shadbala_result",
"vimshottari_timeline",
"narayana_periods",
)
# TASK-rectification-engine-memoization-fix-20260915 §2 measured max
# cross-machine drift of 1.1e-3 on decision_receipt.margin_percent
# (5.2995 vs golden 5.3006). Candidate scores, already rounded to 4
# decimals, drifted by 1 ulp (1.0e-4). Tolerance is the wider of
# abs=2e-3 and rel=5e-4 so 1.1e-3 still passes while a 1e-2 mutation
# (the reverse test below) fails.
GOLDEN_FLOAT_ABS = 2e-3
GOLDEN_FLOAT_REL = 5e-4
def public_score_request() -> dict[str, Any]:
@@ -117,11 +139,102 @@ def _count_calls(monkeypatch, owner: Any, name: str, *, from_engine: bool = Fals
return counter
def _is_json_number(value: Any) -> bool:
return isinstance(value, (int, float)) and not isinstance(value, bool)
def _assert_tiered_equal(actual: Any, expected: Any, *, path: str) -> None:
if isinstance(expected, dict):
assert isinstance(actual, dict), path
assert set(actual) == set(expected), f"{path} keys {set(actual)!r} != {set(expected)!r}"
for key in expected:
_assert_tiered_equal(actual[key], expected[key], path=f"{path}.{key}")
return
if isinstance(expected, list):
assert isinstance(actual, list), path
assert len(actual) == len(expected), f"{path} length {len(actual)} != {len(expected)}"
for index, (left, right) in enumerate(zip(actual, expected)):
_assert_tiered_equal(left, right, path=f"{path}[{index}]")
return
if isinstance(expected, int) and isinstance(actual, int) and not isinstance(expected, bool) and not isinstance(actual, bool):
assert actual == expected, f"{path}: {actual!r} != {expected!r}"
return
if _is_json_number(expected) or _is_json_number(actual):
left = float(actual)
right = float(expected)
tolerance = max(GOLDEN_FLOAT_ABS, GOLDEN_FLOAT_REL * abs(right))
assert math.isclose(left, right, rel_tol=0.0, abs_tol=tolerance), (
f"{path}: {left!r} vs {right!r} exceeds {tolerance}"
)
return
assert actual == expected, f"{path}: {actual!r} != {expected!r}"
def _assert_memoization_payloads(actual: dict[str, Any], expected: dict[str, Any]) -> None:
_assert_tiered_equal(actual["candidate_scores"], expected["candidate_scores"], path="candidate_scores")
_assert_tiered_equal(actual["decision_receipt"], expected["decision_receipt"], path="decision_receipt")
def _event_engine_request() -> dict[str, Any]:
"""Dated events for compute_event_candidate_rows (not the v5 score payload)."""
return {
"birth_date": "1990-01-01",
"start_time": "12:00",
"end_time": "12:02",
"lat": 39.9,
"lon": 116.4,
"tz": 8,
"events": [
{
"id": "00000000-0000-4000-8000-000000000001",
"domain": "education",
"event_kind": "education_start",
"date": "2008-07-01",
"precision": "day",
"summary": "入学",
},
{
"id": "00000000-0000-4000-8000-000000000002",
"domain": "career",
"event_kind": "career_entry",
"date": "2012-06-15",
"precision": "day",
"summary": "入职",
},
],
}
def _contexts_with_layer_cache_cleared(contexts: list[dict[str, Any]]) -> list[dict[str, Any]]:
cleared: list[dict[str, Any]] = []
for context in contexts:
clone = dict(context)
for key in CACHE_LAYER_KEYS:
clone[key] = None
cleared.append(clone)
return cleared
def test_score_candidates_matches_baseline_golden() -> None:
expected = json.loads(GOLDEN_PATH.read_text(encoding="utf-8"))
actual = json.loads(json.dumps(_golden_payload(), ensure_ascii=True))
assert actual["candidate_scores"] == expected["candidate_scores"]
assert actual["decision_receipt"] == expected["decision_receipt"]
_assert_memoization_payloads(actual, expected)
def test_golden_float_shift_of_1e_minus_2_fails() -> None:
expected = json.loads(GOLDEN_PATH.read_text(encoding="utf-8"))
mutated = json.loads(json.dumps(expected))
mutated["candidate_scores"][0]["score"] += 1e-2
with pytest.raises(AssertionError):
_assert_memoization_payloads(expected, mutated)
def test_golden_discrete_field_mismatch_fails() -> None:
expected = json.loads(GOLDEN_PATH.read_text(encoding="utf-8"))
mutated = json.loads(json.dumps(expected))
mutated["candidate_scores"][0]["time"] = "99:99"
with pytest.raises(AssertionError):
_assert_memoization_payloads(expected, mutated)
def test_shadbala_verified_fields_match_with_and_without_birth_minute() -> None:
@@ -372,3 +485,51 @@ def test_compute_event_candidate_rows_reuses_static_context_without_mutating_it(
assert len(rows) == 2
assert [frozenset(context) for context in contexts] == original_keys
assert all("_transit_chart_cache" not in context for context in contexts)
def test_cached_static_context_matches_uncached_fallback(monkeypatch) -> None:
request = _event_engine_request()
cached = [
build_candidate_static_context(request, candidate)
for candidate in _candidate_datetimes(request)
]
assert len(cached) >= 2
assert len(request["events"]) >= 2
for context in cached:
for key in CACHE_LAYER_KEYS:
assert context.get(key) is not None
uncached = _contexts_with_layer_cache_cleared(cached)
for context in uncached:
for key in CACHE_LAYER_KEYS:
assert context[key] is None
calls = _count_calls(monkeypatch, event_engine.shadbala, "calc_shadbala")
rows_cached = compute_event_candidate_rows(request, static_contexts=cached)
cached_calls = calls[0]
rows_uncached = compute_event_candidate_rows(request, static_contexts=uncached)
uncached_calls = calls[0] - cached_calls
assert rows_cached == rows_uncached
assert cached_calls == 0
assert uncached_calls > cached_calls
assert uncached_calls >= len(cached) * len(request["events"])
def test_poisoned_static_cache_diverges_from_live_rows() -> None:
request = _event_engine_request()
cached = [
build_candidate_static_context(request, candidate)
for candidate in _candidate_datetimes(request)
]
rows_cached = compute_event_candidate_rows(request, static_contexts=cached)
poisoned = [dict(context) for context in cached]
poisoned[0]["shadbala_result"] = {
"planets": {
name: {
"sthana_bala": {"total": 0.0},
"drik_bala": 0.0,
"naisargika_bala": 0.0,
}
for name in ("Sun", "Moon", "Mars", "Mercury", "Jupiter", "Venus", "Saturn")
}
}
rows_poisoned = compute_event_candidate_rows(request, static_contexts=poisoned)
assert rows_poisoned != rows_cached