From 69c739fbee3f5af6deed911482d82e49ed429bf7 Mon Sep 17 00:00:00 2001 From: jesse-ux Date: Wed, 14 Oct 2026 06:37:40 +0800 Subject: [PATCH 1/2] =?UTF-8?q?=E4=B8=AA=E4=BA=BA=E9=95=BF=E6=8A=A5?= =?UTF-8?q?=E5=91=8A=E5=90=8C=E7=9B=98=E5=90=8C=E6=97=A5=E9=87=8D=E8=AF=95?= =?UTF-8?q?=E6=94=B9=E4=B8=BA=E7=A7=92=E5=9B=9E?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 完整数据版在版本号明确时复用当天的结果。渲染快捷算法留下但默认关闭，本机只快了大约两成。 Co-Authored-By: Grok (agent loop) --- CHANGELOG.md | 6 + docs/BUG_HISTORY.md | 30 ++ ...PROGRESS-report-reference-perf-20261013.md | 40 ++ docs/tasks/README.md | 2 +- .../report-reference-perf-ab-20261013.md | 61 +++ scripts/calculation_profile_contract.py | 85 +++- scripts/consult_full_data_cache.py | 31 +- scripts/pl9_language_terms.py | 161 ++++++- scripts/professional_report_reference.py | 111 ++++- scripts/report_perf_flags.py | 40 ++ scripts/research/report_reference_perf_ab.py | 428 ++++++++++++++++++ tests/test_report_reference_perf.py | 374 +++++++++++++++ 12 files changed, 1334 insertions(+), 35 deletions(-) create mode 100644 docs/tasks/PROGRESS-report-reference-perf-20261013.md create mode 100644 docs/testing/report-reference-perf-ab-20261013.md create mode 100644 scripts/report_perf_flags.py create mode 100644 scripts/research/report_reference_perf_ab.py create mode 100644 tests/test_report_reference_perf.py diff --git a/CHANGELOG.md b/CHANGELOG.md index d2056bfc..6f5c40b7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,11 @@ # 印度占星 Skill 更新日志 +## 2026-10-14 — 个人长报告引擎出稿提速（同盘同日重试秒回） + +- 同一张盘、同一天、同一个已发布版本，再次生成完整数据版长报告时直接用刚才的结果，本机大约 0.06 秒。第一次生成仍走原来的计算。版本号缺失时不缓存。 +- 中文替换和溯源哈希的快捷算法已留下，但本机只快了大约两成，没有达到开通门槛，默认仍走原算法。 +- Skill 版本不变（BUG-1335、BUG-1336）。 + ## 2026-10-14 — 婚恋评级改成证据档位，KP 判宫和大运年长对齐（待合入 staging） - 婚恋结构指数不再写成婚姻稳定、离婚、再婚或建议婚前咨询。中文报告用中文档位，英文报告用英文档位。配偶星分数按 0 到 1 显示，不再写成百分制。 diff --git a/docs/BUG_HISTORY.md b/docs/BUG_HISTORY.md index add22027..543c8ae7 100644 --- a/docs/BUG_HISTORY.md +++ b/docs/BUG_HISTORY.md @@ -17795,3 +17795,33 @@ - 复发自：无。 - 修复版本：未修复。 +## BUG-1335 | 个人长报告的中文出稿和溯源哈希重复劳动 + +- 状态：investigating +- 首次发现 / 最近更新：2026-10-13 / 2026-10-14 +- 来源：issue #61。任务书 `docs/tasks/TASK-report-reference-perf-20261013.md`。 +- 现象：同一份个人长报告，中文正文要做多遍全文替换；大运补句的正则没有锚点，在长行上反复试；正文里根本没有的术语也会先编译再替换。溯源哈希对每一条「任意层的某个键」都把整包复制一遍。 +- 触发条件：生成完整数据版长报告。不是某一次改坏的，是这些补丁一直往上叠。 +- 根因：替换规则一条条追加，没有先问这句话在不在。哈希按排除路径各走一遍，再单独做键名规范化。 +- 修复：加了等值的预检、预编译，以及哈希的一次遍历。旧代码留在开关后面。本机屏蔽 PyJHora 后，端到端只快了大约两成，没有达到 35% 的开通门槛，所以开关默认关，读者现在仍走原来的出稿速度。 +- 验证：四张虚构盘的中文、英文、事实表、适用性和整份响应逐字节一致。单次哈希中位从 1.115 秒降到 0.402 秒。详见 `docs/testing/report-reference-perf-ab-20261013.md`。 +- 防复发：`scripts/research/report_reference_perf_ab.py` 会在字节不一致时失败；`tests/test_report_reference_perf.py` 用真实引擎包对比开关两边和新旧哈希。 +- 相关记录：BUG-693、BUG-694、BUG-576、BUG-1336。 +- 复发自：无。 +- 修复版本：分支 `codex/report-reference-perf-20261013`，默认未开通，尚未合入 staging。 + +## BUG-1336 | 同一张盘同一天再要长报告，每次都重算 + +- 状态：fixed-pending-verify +- 首次发现 / 最近更新：2026-10-13 / 2026-10-14 +- 来源：issue #61。任务书 `docs/tasks/TASK-report-reference-perf-20261013.md`。 +- 现象：同一输入的长报告每次字节都一样，却没有复用。重试或当天再生成，都要再占一次重计算名额，再等十几秒以上。 +- 触发条件：完整数据版、markdown。其它格式和精简版不在这条里。 +- 根因：咨询卡已经有按键单飞和两天清理，长报告这条生产路径没有接上。 +- 修复：只缓存完整数据版的 markdown。键包含规范化后的出生输入、格式、版本、语言、事实表开关和代码版本号。版本号缺失或是 `unknown` 时不缓存。命中字节与现算相同。失败不落盘。同键并发只算一次。重计算名额逻辑没改，命中只占很短时间。 +- 验证：四张虚构盘命中中位约 0.06 秒，低于 1.5 秒，且与未命中字节相同。新测试覆盖改日期、语言、事实表、候选窗口、版本号都会错过，json 和开关关闭不落盘，并发只建一次，失败后下次会重算。 +- 防复发：`tests/test_report_reference_perf.py` 的缓存用例；A/B 脚本把命中和未命中的 sha256 放在一起比。 +- 相关记录：BUG-576、BUG-1304、BUG-1335、BUG-693。 +- 复发自：无。 +- 修复版本：分支 `codex/report-reference-perf-20261013`，尚未合入 staging。 + diff --git a/docs/tasks/PROGRESS-report-reference-perf-20261013.md b/docs/tasks/PROGRESS-report-reference-perf-20261013.md new file mode 100644 index 00000000..c719bf99 --- /dev/null +++ b/docs/tasks/PROGRESS-report-reference-perf-20261013.md @@ -0,0 +1,40 @@ +# PROGRESS · 长报告出稿提速（2026-10-14） + +- 分支：`codex/report-reference-perf-20261013` +- 实际基线：`origin/staging` `66cd60081f37a57680274633fcb3fb942d01bb2f`（任务书写的 `77057a819`、计划评论里的 `2994588ad` 都更早，按当前 staging 开工） +- `scripts/jyotish_api_server.py`：10335 行，零改动 +- 历史最大号在本基线上是 BUG-1322。BUG-1323～1334 已被其它未合入任务书占用。本单用任务书指定的 BUG-1335、BUG-1336。 + +## 做了什么 + +1. 两个开关在调用时读环境变量。渲染快捷路径默认关，同盘缓存默认开。 +2. 中文和英文的大运密度补句：正则预编译，行内先看必出现的字面量，没有才不搜索；向前看的五行只在可能命中时才拼。术语替换：正文里没有这个词就原样返回，编译结果按词缓存。 +3. 溯源哈希在开关打开时，一次遍历同时删掉 `**.键` 并做键名规范化。覆盖范围和排除集没变。没有跳过中间那次绑定，因为那会改 `format=json` 的键顺序。 +4. 咨询卡缓存的单飞和两天清理多了一个子目录参数，咨询卡自己的键和目录不变。长报告只在完整数据版 + markdown、开关开、并且 `GITHUB_SHA` 是具体版本号时，才把整份响应放进 `scratch/local/report_reference_cache/`。查缓存发生在重计算之前。命中打一行不含资料的日志。 + +## 自测 + +| 检查 | 命令 | 结果 | +| --- | --- | --- | +| 任务书列出的 pytest | `.venv\Scripts\python.exe -m pytest` 后接任务书第 8 节那一串文件 `-q` | 退出码 0。缓存默认改成开之后又跑了一遍，仍是退出码 0 | +| 屏蔽 PyJHora 的 A/B | `python scripts/research/report_reference_perf_ab.py --mask-pyjhora` | 四张盘五字段 sha 相同；端到端下降 20%～23%，未达 35%；缓存命中中位约 0.06 秒 | +| 同一份解盘的 json / reference | 同上脚本 `--side-only` | 退出码 0。时钟钉死且复用同一次解盘后，两版 sha 相同 | +| 不屏蔽 | 未跑 | 本机没有 PyJHora，和屏蔽走同一条导入失败路径 | + +数字见 `docs/testing/report-reference-perf-ab-20261013.md`。 + +## 改既有断言 + +无。原值 / 新值 / 原因：没有改旧断言。 + +## 未做 + +- 整篇替换逐条预检（让步顺序里最先砍的一项）。中文渲染只降了 20%～28%，不到 40%，没有为了过线改输出。 +- 跳过中间绑定。 +- 关掉没人读的质量门。任务书写明这要另开单，本单不动。 +- 旧路径退役。开关还在。 + +## 默认值 + +- `JYOTISH_REPORT_RENDER_FAST` 默认关。字节一致，内存还降了约 12%，但端到端没到 35%。 +- `JYOTISH_REPORT_REFERENCE_CACHE` 默认开。命中与未命中字节一致，命中中位约 0.06 秒。版本号缺失或是 `unknown` 时不缓存。 diff --git a/docs/tasks/README.md b/docs/tasks/README.md index 58b326a6..3f229f6d 100644 --- a/docs/tasks/README.md +++ b/docs/tasks/README.md @@ -471,7 +471,7 @@ | `TASK-node-runtime-resources-20261013.md` | — | 报告 worker abort 监听泄漏、本地 PG 适配层往返与元数据缓存、无上限内存表、用户端会话读缓存（BUG-1326～1328、1346，PG 与缓存部分 A/B） | 待领取 | `codex/node-runtime-resources-20261013` | | `TASK-home-sidebar-rerender-20261013.md` | — | 流式回答时侧栏每帧整栏重渲染（BUG-1329，A/B） | 待领取 | `codex/home-sidebar-rerender-20261013` | | `TASK-api-disk-cache-retention-20261013.md` | — | API 磁盘缓存只进不出、VedAstro 审计文件留存出生资料、失败快照久缓存、缓存键规范化（BUG-1330～1332，A/B） | 待领取 | `codex/api-disk-cache-retention-20261013` | -| `TASK-report-reference-perf-20261013.md` | — | 长报告出稿提速：术语替换命中预检、哈希单次遍历、同盘复用（BUG-1335～1336，A/B） | 待领取 | `codex/report-reference-perf-20261013` | +| `TASK-report-reference-perf-20261013.md` | `PROGRESS-report-reference-perf-20261013.md` | 长报告出稿提速：术语替换命中预检、哈希单次遍历、同盘复用（BUG-1335～1336，A/B） | 待验收 | `codex/report-reference-perf-20261013` | | `TASK-consult-prep-parallel-20261013.md` | — | 普通对话发问前准备步骤并行、模型版本短缓存、多领域并发开关（BUG-1337～1339，A/B） | 待领取 | `codex/consult-prep-parallel-20261013` | | `TASK-frontend-dead-code-20261013.md` | — | 前端死代码 17 文件与无调用方接口删除，含未登录重计算接口（BUG-1342） | 待领取 | `codex/frontend-dead-code-20261013` | | `TASK-api-chart-subprocess-gate-20261013.md` | — | 合盘不带 skip 串行排盘、报告无效子进程、名额轻重混用（只模拟）（BUG-1333～1334，A/B） | 待领取（排队：api-disk-cache-retention + report-reference-perf） | `codex/api-chart-subprocess-gate-20261013` | diff --git a/docs/testing/report-reference-perf-ab-20261013.md b/docs/testing/report-reference-perf-ab-20261013.md new file mode 100644 index 00000000..2da1f621 --- /dev/null +++ b/docs/testing/report-reference-perf-ab-20261013.md @@ -0,0 +1,61 @@ +# 长报告出稿 A/B（2026-10-14） + +Windows 本机，Python 3.11.7，`Windows-10-10.0.26200-SP0`。屏蔽 PyJHora（生产镜像不装这个库）。本机本身也没有 PyJHora，所以没有再跑一轮不屏蔽的重复计时。计时开了 tracemalloc，绝对秒数比任务书里的 Linux 基线高，只比较同一进程里的 A 和 B。 + +A = 两个开关关。B = 两个开关开。四张虚构盘，每组 3 次。生产请求是完整数据版、markdown、中英都要、带事实表。没有模型钥匙。 + +## 端到端（未命中，秒） + +| 盘 | A 三次 | A 中位 | B 三次 | B 中位 | 下降 | +| --- | --- | --- | --- | --- | --- | +| day | 71.31 / 64.03 / 64.01 | 64.03 | 51.05 / 50.77 / 51.74 | 51.05 | 20.3% | +| night | 66.25 / 66.11 / 66.66 | 66.25 | 53.32 / 52.57 / 50.43 | 52.57 | 20.7% | +| south | 66.03 / 66.00 / 62.69 | 66.00 | 49.97 / 52.79 / 52.68 | 52.68 | 20.2% | +| section2 | 71.01 / 108.37 / 74.19 | 74.19 | 57.42 / 57.12 / 71.80 | 57.42 | 22.6% | + +门槛是未命中端到端下降至少 35%。四张盘都没到。渲染快捷路径默认关闭。 + +## 分段中位数（秒） + +| 盘 | 中文渲染 A → B | 英文渲染 A → B | 建包 A → B | +| --- | --- | --- | --- | +| day | 23.55 → 17.04（−27.6%） | 8.27 → 8.67 | 22.55 → 16.08 | +| night | 23.42 → 17.19（−26.6%） | 8.75 → 8.41 | 23.70 → 17.21 | +| south | 23.89 → 17.39（−27.2%） | 8.66 → 8.74 | 22.90 → 16.87 | +| section2 | 22.26 → 17.80（−20.0%） | 8.14 → 8.54 | 31.24 → 21.08 | + +中文渲染门槛是下降至少 40%。没到。没有继续加整篇替换的预检。 + +## 内存与字节 + +tracemalloc 峰值：B 约为 A 的 0.87 到 0.88，没有升高。 + +五份输出（整份响应、中文、英文、事实表、适用性）A、B 未命中、缓存命中的 sha256 全部相同。 + +| 盘 | 响应字节 | 响应 sha256 前 12 位 | +| --- | --- | --- | +| day | 2289867 | 5b542359d606 | +| night | 2321638 | 89baa960c054 | +| south | 2314369 | 250bf1ab246e | +| section2 | 2238092 | 1d7afd481b1a | + +## 同盘缓存（命中，秒） + +| 盘 | 三次 | 中位 | +| --- | --- | --- | +| day | 0.059 / 0.054 / 0.057 | 0.057 | +| night | 0.057 / 0.054 / 0.058 | 0.057 | +| south | 0.059 / 0.054 / 0.057 | 0.057 | +| section2 | 0.063 / 0.066 / 0.057 | 0.063 | + +命中字节与未命中相同，中位数远低于 1.5 秒。缓存开关默认开。`GITHUB_SHA` 缺失或为 `unknown` 时仍然关。 + +## 单次溯源哈希（交付包，秒） + +关：1.132 / 1.099 / 1.115，中位 1.115。开：0.402 / 0.466 / 0.388，中位 0.402。下降 64%，过了 50% 那一档。它跟渲染快捷路径用同一个开关，所以默认也是关。 + +## json 与 reference 版 + +同一份解盘、时钟钉死后，`format=json` 和 reference 版 markdown 在快捷路径关与开时 sha256 相同（`aa1352619b7a`、`cbadf95f549a`）。对比的是整份响应，没有丢掉字段。 + +连续两次重新解盘会对不齐：有的特质列表挂在类上，第二次会再追加。那不是这条快捷路径改出来的。生产要的完整数据 markdown 在多次重算里仍然逐字节相同。 diff --git a/scripts/calculation_profile_contract.py b/scripts/calculation_profile_contract.py index 7a7f5871..b9f6e0b6 100755 --- a/scripts/calculation_profile_contract.py +++ b/scripts/calculation_profile_contract.py @@ -486,15 +486,26 @@ def _drop_dotted_path(payload: Any, path: tuple[str, ...]) -> Any: return cloned -def result_payload_for_binding( +def _render_fast() -> bool: + try: + from scripts.report_perf_flags import report_render_fast_enabled + except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution + from report_perf_flags import report_render_fast_enabled + return report_render_fast_enabled() + + +def _excluded_binding_top(binding_scope: dict[str, Any]) -> set[str]: + return RESULT_BINDING_ENVELOPE_KEYS | { + str(key) for key in (binding_scope.get("excluded_top_keys") or ()) + } + + +def _result_payload_for_binding_legacy( result: dict[str, Any], scope: dict[str, Any] | None = None, ) -> dict[str, Any]: - """Business payload that participates in result_hash.""" binding_scope = scope if isinstance(scope, dict) else default_result_binding_scope() - excluded_top = RESULT_BINDING_ENVELOPE_KEYS | { - str(key) for key in (binding_scope.get("excluded_top_keys") or ()) - } + excluded_top = _excluded_binding_top(binding_scope) payload = {key: value for key, value in result.items() if key not in excluded_top} for raw_path in binding_scope.get("excluded_paths") or (): parts = tuple(part for part in str(raw_path).split(".") if part) @@ -503,17 +514,73 @@ def result_payload_for_binding( return payload +def _drop_star_keys_and_canonicalize(payload: Any, star_keys: frozenset[str]) -> Any: + """Drop ``**.key`` names and canonicalize keys in one walk. + + String keys use the same inequality as ``_drop_key_recursive``. Non-string + keys are kept, then renamed with the same rule as ``canonicalize_result_payload``. + """ + if isinstance(payload, dict): + normalized: dict[str, Any] = {} + for key, child in payload.items(): + if isinstance(key, str): + if key in star_keys: + continue + normalized_key = key + else: + normalized_key = f"__{type(key).__name__}__:{key}" + normalized[normalized_key] = _drop_star_keys_and_canonicalize(child, star_keys) + return normalized + if isinstance(payload, (list, tuple)): + return [_drop_star_keys_and_canonicalize(item, star_keys) for item in payload] + return payload + + +def _result_payload_for_binding_fast( + result: dict[str, Any], + scope: dict[str, Any] | None = None, +) -> dict[str, Any]: + binding_scope = scope if isinstance(scope, dict) else default_result_binding_scope() + excluded_top = _excluded_binding_top(binding_scope) + payload = {key: value for key, value in result.items() if key not in excluded_top} + star_keys: set[str] = set() + for raw_path in binding_scope.get("excluded_paths") or (): + parts = tuple(part for part in str(raw_path).split(".") if part) + if len(parts) == 2 and parts[0] == "**": + star_keys.add(parts[1]) + continue + if parts: + payload = _drop_dotted_path(payload, parts) + return _drop_star_keys_and_canonicalize(payload, frozenset(star_keys)) + + +def result_payload_for_binding( + result: dict[str, Any], + scope: dict[str, Any] | None = None, +) -> dict[str, Any]: + """Business payload that participates in result_hash.""" + if _render_fast(): + return _result_payload_for_binding_fast(result, scope) + return _result_payload_for_binding_legacy(result, scope) + + def hash_bound_result( result: dict[str, Any], input_hash: str, scope: dict[str, Any] | None = None, ) -> str: """SHA-256 of the canonical bound payload. Does not mutate ``result``.""" - encoded = json.dumps( - canonicalize_result_payload({ + payload = result_payload_for_binding(result, scope) + if _render_fast(): + # The fast payload is already canonical, including the star-key drop. + canonical = {"input_hash": input_hash, "result": payload} + else: + canonical = canonicalize_result_payload({ "input_hash": input_hash, - "result": result_payload_for_binding(result, scope), - }), + "result": payload, + }) + encoded = json.dumps( + canonical, ensure_ascii=False, sort_keys=True, separators=(",", ":"), diff --git a/scripts/consult_full_data_cache.py b/scripts/consult_full_data_cache.py index 43ffdd34..32ef94d9 100644 --- a/scripts/consult_full_data_cache.py +++ b/scripts/consult_full_data_cache.py @@ -57,8 +57,8 @@ def _release_warm_slot() -> None: _warm_inflight -= 1 -def cache_dir() -> Path: - path = ROOT / "scratch" / "local" / "consult_full_data_cache" +def cache_dir(name: str = "consult_full_data_cache") -> Path: + path = ROOT / "scratch" / "local" / name path.mkdir(parents=True, exist_ok=True) return path @@ -118,15 +118,15 @@ def cache_key(identity: dict[str, Any]) -> str: return hashlib.sha256(canonical.encode("utf-8")).hexdigest() -def _cache_path(identity: dict[str, Any]) -> Path: - return cache_dir() / f"{cache_key(identity)}.json" +def _cache_path(identity: dict[str, Any], name: str = "consult_full_data_cache") -> Path: + return cache_dir(name) / f"{cache_key(identity)}.json" -def prune_cache(now: float | None = None) -> int: +def prune_cache(now: float | None = None, *, name: str = "consult_full_data_cache") -> int: """Drop cache files last written more than two days ago.""" cutoff = (time.time() if now is None else now) - KEEP_SECONDS removed = 0 - folder = cache_dir() + folder = cache_dir(name) for path in folder.glob("*.json"): try: if path.stat().st_mtime < cutoff: @@ -147,15 +147,21 @@ def get_or_build( builder: PacketBuilder | None = None, *, clock: Callable[[], float] | None = None, + cache_name: str = "consult_full_data_cache", + json_default: Callable[..., Any] | None = None, ) -> dict[str, Any]: - """Return the packet. Concurrent callers for one key share one build.""" - prune_cache() + """Return the packet. Concurrent callers for one key share one build. + + ``cache_name`` selects the scratch subdirectory. The consult card keeps the + default. A non-default name does not change the consult key or its files. + """ + prune_cache(name=cache_name) now = clock or time.perf_counter started = now() - path = _cache_path(identity) + path = _cache_path(identity, cache_name) if path.is_file(): return _read_cached(path, started, now) - key = cache_key(identity) + key = f"{cache_name}\n{cache_key(identity)}" with _flights_guard: flight = _flights.get(key) owner = flight is None @@ -177,7 +183,10 @@ def get_or_build( builder = build_full_data_packet packet = builder(identity) temporary = path.with_suffix(".json.tmp") - temporary.write_text(json.dumps(packet, ensure_ascii=False, separators=(",", ":")), encoding="utf-8") + dump_kwargs: dict[str, Any] = {"ensure_ascii": False, "separators": (",", ":")} + if json_default is not None: + dump_kwargs["default"] = json_default + temporary.write_text(json.dumps(packet, **dump_kwargs), encoding="utf-8") temporary.replace(path) elapsed = now() - started return { diff --git a/scripts/pl9_language_terms.py b/scripts/pl9_language_terms.py index 4e90b683..92c58144 100644 --- a/scripts/pl9_language_terms.py +++ b/scripts/pl9_language_terms.py @@ -7,9 +7,15 @@ It only changes reader-facing labels; it does not alter calculation payloads. """ from __future__ import annotations +import functools import re from typing import Any +try: + from scripts.report_perf_flags import report_render_fast_enabled +except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution + from report_perf_flags import report_render_fast_enabled + # Bilingual planet tokens from yoga combination text, longest first. # Dignity is glued on with no space ("木星Jupiterexalted"), so the long form @@ -131,6 +137,11 @@ def translate_zh_glued_phrases(text: str) -> str: return text +@functools.lru_cache(maxsize=512) +def _natural_language_term_pattern(raw: str, plural: str) -> re.Pattern: + return re.compile(rf"(? str: """Replace one glossary term in prose only. @@ -141,7 +152,14 @@ def replace_natural_language_term(text: str, raw: str, public: str) -> str: if not text or not raw: return text plural = "s?" if re.search(r"[A-Za-z]$", raw) and not raw.endswith("s") else "" - pattern = re.compile(rf"(? str: ) +# Every AD/PD hit contains these literals. Patterns stay case-sensitive. +_ZH_AD_DENSITY_RE = re.compile( + r"(.+?)主大运中的(.+?)子运，会把(.+?)的自然主题带入当前阶段。" +) +_ZH_PD_DENSITY_RE = re.compile( + r"(.+?)子运中的(.+?)次子运，会在(.+?)背景下呈现(.+?)的短周期结果。" +) +_ZH_AD_ANCHORS = ("主大运中的", "子运，会把", "的自然主题带入当前阶段。") +_ZH_PD_ANCHORS = ("次子运，会在", "背景下呈现", "的短周期结果。") +_EN_MD_HEADING_RE = re.compile(r"^#{2,4}\s+(.+?\bMaha Dasha\b.*)$") +_EN_AD_DENSITY_RE = re.compile( + r"The Antar Dasha of (.+?) activates (.+?)'s natural themes inside the broader Maha Dasha of (.+?)\." +) +_EN_PD_DENSITY_RE = re.compile( + r"In the Pratyantar Dasha of (.+?) in the Antar Dasha of (.+?), (.+?)'s significations give the short-period result inside the (.+?) background\." +) +_EN_AD_ANCHORS = ( + "The Antar Dasha of ", + " activates ", + "'s natural themes inside the broader Maha Dasha of ", +) +_EN_PD_ANCHORS = ( + "In the Pratyantar Dasha of ", + " in the Antar Dasha of ", + "'s significations give the short-period result inside the ", + " background.", +) + + +def _line_has_anchors(line: str, anchors: tuple[str, ...]) -> bool: + return all(anchor in line for anchor in anchors) + + +def _thicken_zh_dasha_density_fast(markdown: str) -> str: + """Same AD/PD inserts as the legacy scan, without searching inert lines.""" + lines = markdown.splitlines() + out: list[str] = [] + for index, line in enumerate(lines): + out.append(line) + # Both patterns are case-sensitive and every hit contains these literals. + ad_possible = _line_has_anchors(line, _ZH_AD_ANCHORS) + pd_possible = _line_has_anchors(line, _ZH_PD_ANCHORS) + if not ad_possible and not pd_possible: + continue + lookahead = "\n".join(lines[index + 1:index + 6]) + if "这一子运不是单独结论" in lookahead or "这个次子运只承担短周期细化" in lookahead: + continue + if ad_possible: + ad_match = _ZH_AD_DENSITY_RE.search(line) + if ad_match: + md_lord, ad_lord, theme_lord = ad_match.groups() + out.extend([ + "", + ( + f"这一子运不是单独结论，而是把{theme_lord}的自然象征放进{md_lord}主大运的背景里筛选。" + f"阅读时先看{ad_lord}本身的落宫、守护宫、尊贵状态与星宿，再看它和主大运星之间是否互相支持。" + ), + "", + ( + "若同一宫位或同一行星在分盘、Ashtakavarga、年度盘和行运中重复出现，" + "该主题的可见度会提高；若这些层彼此冲突，则应把结果视为阶段性倾向，而不是孤立断语。" + ), + ]) + continue + if pd_possible: + pd_match = _ZH_PD_DENSITY_RE.search(line) + if pd_match: + ad_lord, pd_lord, md_lord, theme_lord = pd_match.groups() + out.extend([ + "", + ( + f"这个次子运只承担短周期细化：{pd_lord}会把{theme_lord}的具体征象带入{ad_lord}子运，" + f"并受{md_lord}主背景限制。" + ), + "", + ( + "因此这里优先用于判断事情推进的节奏、触发点和轻重缓急；实际领域仍需回到本命落宫、" + "分盘重复、年度盘和当前行运共同核对。" + ), + ]) + continue + return "\n".join(out) + + def _thicken_zh_dasha_density(markdown: str) -> str: """Restore reader-facing density for translated AD/PD prose. This adds reading instructions tied to already-present fields. It does not introduce new predictions or alter dates. """ + if report_render_fast_enabled(): + return _thicken_zh_dasha_density_fast(markdown) lines = markdown.splitlines() out: list[str] = [] for index, line in enumerate(lines): @@ -941,9 +1045,64 @@ def normalize_en_markdown_terms(markdown: str) -> str: return apply_en_residual_han(_thicken_en_dasha_density(text)) +def _thicken_en_dasha_density_fast(markdown: str) -> str: + """Same English AD inserts as the legacy scan, without searching inert lines.""" + lines = markdown.splitlines() + out: list[str] = [] + current_md = None + ad_context_added = False + for index, line in enumerate(lines): + md_heading = _EN_MD_HEADING_RE.match(line) + if md_heading: + next_md = md_heading.group(1) + if next_md != current_md: + current_md = next_md + ad_context_added = False + out.append(line) + # Both patterns are case-sensitive and every hit contains these literals. + ad_possible = _line_has_anchors(line, _EN_AD_ANCHORS) + pd_possible = _line_has_anchors(line, _EN_PD_ANCHORS) + if not ad_possible and not pd_possible: + continue + lookahead = "\n".join(lines[index + 1:index + 6]) + if ( + "This Antar Dasha is not read as a standalone verdict" in lookahead + or "This Pratyantar Dasha is the short-cycle refinement" in lookahead + ): + continue + if ad_possible: + ad_match = _EN_AD_DENSITY_RE.search(line) + if ad_match and not ad_context_added: + ad_lord, theme_lord, md_lord = ad_match.groups() + out.extend([ + "", + ( + f"This Antar Dasha is not read as a standalone verdict. It places {theme_lord}'s " + f"natural significations inside the background of the {md_lord} Maha Dasha, then " + f"filters them through {ad_lord}'s house placement, house ownership, dignity, " + "nakshatra lord, conjunctions, and divisional-chart repetition." + ), + "", + ( + "When the same planet or house repeats through divisional charts, Ashtakavarga, " + "annual factors, and transits, the theme becomes more visible. When those layers " + "conflict, the result should be read as a staged tendency rather than an isolated statement." + ), + ]) + ad_context_added = True + continue + if pd_possible and _EN_PD_DENSITY_RE.search(line): + # The row already contains the concrete PD lord and timing context. + # Keep it, but do not append the same methodology paragraph per node. + continue + return "\n".join(out) + + def _thicken_en_dasha_density(markdown: str) -> str: """Add chapter-level AD context without repeating boilerplate on every PD row.""" + if report_render_fast_enabled(): + return _thicken_en_dasha_density_fast(markdown) lines = markdown.splitlines() out: list[str] = [] current_md = None diff --git a/scripts/professional_report_reference.py b/scripts/professional_report_reference.py index 5167eaca..d4ccc8b8 100644 --- a/scripts/professional_report_reference.py +++ b/scripts/professional_report_reference.py @@ -8,6 +8,9 @@ calculation, and delegates packet assembly/rendering to ``jyotish_engine``. from __future__ import annotations import copy +import os +import sys +import time from datetime import datetime from importlib import import_module from types import SimpleNamespace @@ -32,6 +35,11 @@ except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution path from reader_dasha_applicability import render_dasha_applicability from report_density_packet import report_density_packet +try: + from scripts.report_perf_flags import report_reference_cache_enabled +except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution path + from report_perf_flags import report_reference_cache_enabled + class ProfessionalReportReferenceInputError(ValueError): """The professional-reference request is outside the public contract.""" @@ -187,18 +195,49 @@ def _export_args(birth: dict[str, Any]) -> SimpleNamespace: return SimpleNamespace(**payload) -def build_professional_report_reference(handler, body: dict[str, Any], *, engine=None) -> dict[str, Any]: - """Build one JSON or Markdown response from one reused full-reading result.""" - if not isinstance(body, dict): - raise ProfessionalReportReferenceInputError("request body must be an object") - output_format = _normalize_format(body.get("format")) - edition = _normalize_edition(body.get("edition")) - packs = _normalize_packs(body.get("packs")) - languages = _normalize_languages(body.get("languages")) - if languages and (output_format != "markdown" or edition != FULL_DATA_EDITION): - raise ProfessionalReportReferenceInputError("languages requires format markdown and edition full_data") +def _load_report_cache(): + try: + from scripts import consult_full_data_cache as cache + except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution path + import consult_full_data_cache as cache + return cache - birth = handler._high_rigor_birth_payload(body) + +def _report_cache_sha() -> str: + return os.environ.get("GITHUB_SHA") or "" + + +def _report_cache_identity( + birth: dict[str, Any], + output_format: str, + edition: str, + packs: list[str], + languages: list[str], + include_fact_tables: bool, + sha: str, +) -> dict[str, Any]: + return { + "birth": birth, + "format": output_format, + "edition": edition, + "packs": list(packs), + "languages": list(languages), + "include_fact_tables": include_fact_tables, + "report_version": REPORT_VERSION_FULL, + "github_sha": sha, + } + + +def _assemble_professional_report_reference( + handler, + birth: dict[str, Any], + output_format: str, + edition: str, + packs: list[str], + languages: list[str], + include_fact_tables: bool, + engine, +) -> dict[str, Any]: full_reading = handler._compute_full_reading_for_thematic(birth) resolved_engine = engine or _load_engine() try: @@ -217,7 +256,7 @@ def build_professional_report_reference(handler, body: dict[str, Any], *, engine if edition == FULL_DATA_EDITION: # English renders from its own deep copy first, so nothing the Chinese # renderer does to the packet can reach it (and vice versa). - english = _english_edition(packet, body.get("include_fact_tables") is True) if "en" in languages else {} + english = _english_edition(packet, include_fact_tables) if "en" in languages else {} markdown = render_full_data_markdown(packet) else: markdown = resolved_engine.render_pl9_markdown(packet) @@ -228,7 +267,7 @@ def build_professional_report_reference(handler, body: dict[str, Any], *, engine "markdown": markdown, **({"fact_table_packet": report_density_packet(packet), "reader_dasha_applicability": clean_reader_appendix_markdown(render_dasha_applicability(packet))} - if body.get("include_fact_tables") is True else {}), + if include_fact_tables else {}), **(english if edition == FULL_DATA_EDITION and "en" in languages else {}), } return { @@ -236,3 +275,49 @@ def build_professional_report_reference(handler, body: dict[str, Any], *, engine "edition": edition, "report": packet, } + + +def build_professional_report_reference(handler, body: dict[str, Any], *, engine=None) -> dict[str, Any]: + """Build one JSON or Markdown response from one reused full-reading result.""" + if not isinstance(body, dict): + raise ProfessionalReportReferenceInputError("request body must be an object") + output_format = _normalize_format(body.get("format")) + edition = _normalize_edition(body.get("edition")) + packs = _normalize_packs(body.get("packs")) + languages = _normalize_languages(body.get("languages")) + if languages and (output_format != "markdown" or edition != FULL_DATA_EDITION): + raise ProfessionalReportReferenceInputError("languages requires format markdown and edition full_data") + + birth = handler._high_rigor_birth_payload(body) + include_fact_tables = body.get("include_fact_tables") is True + sha = _report_cache_sha() + use_cache = ( + report_reference_cache_enabled() + and output_format == "markdown" + and edition == FULL_DATA_EDITION + and sha not in ("", "unknown") + ) + if not use_cache: + return _assemble_professional_report_reference( + handler, birth, output_format, edition, packs, languages, include_fact_tables, engine, + ) + identity = _report_cache_identity( + birth, output_format, edition, packs, languages, include_fact_tables, sha, + ) + started = time.perf_counter() + + def builder(_identity: dict[str, Any]) -> dict[str, Any]: + return _assemble_professional_report_reference( + handler, birth, output_format, edition, packs, languages, include_fact_tables, engine, + ) + + cached = _load_report_cache().get_or_build( + identity, + builder, + cache_name="report_reference_cache", + json_default=str, + ) + elapsed_ms = int(round((time.perf_counter() - started) * 1000)) + state = "hit" if cached["hit"] else "miss" + print(f"[report-reference-cache] {state} elapsed_ms={elapsed_ms}", file=sys.stderr) + return cached["packet"] diff --git a/scripts/report_perf_flags.py b/scripts/report_perf_flags.py new file mode 100644 index 00000000..2278bf7d --- /dev/null +++ b/scripts/report_perf_flags.py @@ -0,0 +1,40 @@ +"""Call-time switches for long-report rendering and same-chart reuse. + +Values are read on every call so one process can measure the old path and the +new path back to back. The old path stays in place when a switch is off. +""" + +from __future__ import annotations + +import os + +# Masked Windows A/B on 2026-10-14. Render shortcuts are byte-identical and use +# less memory, but the end-to-end drop was 20–23%, short of the 35% gate, so +# the fast path stays off. Cache hits matched the miss bytes and returned in +# about 0.06s, under the 1.5s gate. A missing or unknown GITHUB_SHA still +# disables the cache at the call site. +RENDER_FAST_DEFAULT = False +REFERENCE_CACHE_DEFAULT = True + +_ON = {"1", "true", "yes", "on"} + + +def _enabled(name: str, default: bool) -> bool: + raw = os.environ.get(name) + if raw is None or not raw.strip(): + return default + return raw.strip().lower() in _ON + + +def report_render_fast_enabled() -> bool: + """True when the equivalent render and hash shortcuts may run.""" + return _enabled("JYOTISH_REPORT_RENDER_FAST", RENDER_FAST_DEFAULT) + + +def report_reference_cache_enabled() -> bool: + """True when a full_data markdown response may be reused from disk. + + A missing or ``unknown`` ``GITHUB_SHA`` still disables the cache at the + call site. This function only reads the switch. + """ + return _enabled("JYOTISH_REPORT_REFERENCE_CACHE", REFERENCE_CACHE_DEFAULT) diff --git a/scripts/research/report_reference_perf_ab.py b/scripts/research/report_reference_perf_ab.py new file mode 100644 index 00000000..fb2210e2 --- /dev/null +++ b/scripts/research/report_reference_perf_ab.py @@ -0,0 +1,428 @@ +"""A/B timing for the professional long report. + +A is both switches off. B is both on. Same process, fictional charts only. +A field sha mismatch exits 1 and prints a short diff context, not the report. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import platform +import sys +import tempfile +import time +import tracemalloc +from pathlib import Path +from types import SimpleNamespace + + +ROOT = Path(__file__).resolve().parents[2] +for _folder in (ROOT / "scripts", ROOT / "jyotish_vedic", ROOT): + _text = str(_folder) + if _text not in sys.path: + sys.path.insert(0, _text) + +PRODUCTION_BODY = { + "format": "markdown", + "edition": "full_data", + "include_fact_tables": True, + "languages": ["zh", "en"], +} +SECTION2_CHART = { + "year": 1990, + "month": 1, + "day": 1, + "hour": 12, + "minute": 0, + "second": 0, + "lat": 28.6139, + "lon": 77.2090, + "tz": 5.5, + "node_mode": "mean", + "ayanamsa": "lahiri", + "today": "2026-10-13", + "transit_date": "2026-10-13", + "target_year": 2026, + "age": 36, + "birth_time_accuracy": "confirmed", +} +FIELDS = ( + "response", + "markdown", + "markdown_en", + "fact_table_packet", + "reader_dasha_applicability", +) + + +def _mask_pyjhora() -> str: + root = Path(tempfile.mkdtemp(prefix="nojhora-")) + package = root / "jhora" + package.mkdir() + (package / "__init__.py").write_text('raise ImportError("masked")\n', encoding="utf-8") + sys.path.insert(0, str(root)) + prior = os.environ.get("PYTHONPATH", "") + os.environ["PYTHONPATH"] = str(root) + (os.pathsep + prior if prior else "") + for name in list(sys.modules): + if name == "jhora" or name.startswith("jhora."): + del sys.modules[name] + return str(root) + + +def _sha(text: str) -> str: + return hashlib.sha256(text.encode("utf-8")).hexdigest() + + +def _canonical(value) -> str: + if isinstance(value, str): + return value + return json.dumps(value, ensure_ascii=False, default=str) + + +def _field_texts(result: dict) -> dict[str, str]: + whole = json.dumps(result, ensure_ascii=False, default=str) + return { + "response": whole, + "markdown": _canonical(result.get("markdown", "")), + "markdown_en": _canonical(result.get("markdown_en", "")), + "fact_table_packet": _canonical(result.get("fact_table_packet")), + "reader_dasha_applicability": _canonical(result.get("reader_dasha_applicability", "")), + } + + +def _field_shas(result: dict) -> dict[str, str]: + return {name: _sha(text) for name, text in _field_texts(result).items()} + + +def _diff_context(left: str, right: str) -> str: + limit = min(len(left), len(right)) + index = next((i for i in range(limit) if left[i] != right[i]), limit) + start = max(0, index - 40) + stop = index + 40 + return ( + f"at {index} len {len(left)} vs {len(right)} " + f"A={left[start:stop]!r} B={right[start:stop]!r}" + ) + + +def _median(values: list[float]) -> float: + ordered = sorted(values) + return ordered[len(ordered) // 2] + + +class _Bucket: + def __init__(self) -> None: + self.cmd = 0.0 + self.packet = 0.0 + self.zh = 0.0 + self.en = 0.0 + self.hash = 0.0 + + def reset(self) -> None: + self.cmd = self.packet = self.zh = self.en = self.hash = 0.0 + + def as_dict(self) -> dict[str, float]: + return { + "cmd_full_reading_s": round(self.cmd, 4), + "packet_s": round(self.packet, 4), + "zh_render_s": round(self.zh, 4), + "en_render_s": round(self.en, 4), + "hash_s": round(self.hash, 4), + } + + +def _install_timers(bucket: _Bucket) -> None: + import scripts.calculation_profile_contract as contract + import scripts.full_report_quality_gate as gate + import scripts.jyotish_engine as engine + import scripts.professional_report_reference as reference + + orig_packet = engine.build_professional_report_reference_packet + + def timed_packet(*args, **kwargs): + started = time.perf_counter() + try: + return orig_packet(*args, **kwargs) + finally: + bucket.packet += time.perf_counter() - started + + engine.build_professional_report_reference_packet = timed_packet + orig_render = reference.render_full_data_markdown + + def timed_render(packet, *args, **kwargs): + started = time.perf_counter() + try: + return orig_render(packet, *args, **kwargs) + finally: + elapsed = time.perf_counter() - started + if isinstance(packet, dict) and packet.get("report_language") == "en": + bucket.en += elapsed + else: + bucket.zh += elapsed + + reference.render_full_data_markdown = timed_render + orig_hash = contract.hash_bound_result + + def timed_hash(*args, **kwargs): + started = time.perf_counter() + try: + return orig_hash(*args, **kwargs) + finally: + bucket.hash += time.perf_counter() - started + + contract.hash_bound_result = timed_hash + gate.hash_bound_result = timed_hash + + +def _set_flags(fast: bool, cache: bool) -> None: + os.environ["JYOTISH_REPORT_RENDER_FAST"] = "1" if fast else "0" + os.environ["JYOTISH_REPORT_REFERENCE_CACHE"] = "1" if cache else "0" + + +class _Handler: + def __init__(self, birth: dict, bucket: _Bucket) -> None: + self.birth = birth + self.bucket = bucket + + def _high_rigor_birth_payload(self, _body: dict) -> dict: + return dict(self.birth) + + def _compute_full_reading_for_thematic(self, birth: dict): + from scripts.jyotish_engine import cmd_full_reading + + started = time.perf_counter() + try: + return cmd_full_reading(SimpleNamespace(**birth)) + finally: + self.bucket.cmd += time.perf_counter() - started + + +def _run_once(birth: dict, body: dict, bucket: _Bucket, handler=None) -> dict: + from scripts.professional_report_reference import build_professional_report_reference + + bucket.reset() + tracemalloc.start() + started = time.perf_counter() + result = build_professional_report_reference(handler or _Handler(birth, bucket), body) + elapsed = time.perf_counter() - started + _current, peak = tracemalloc.get_traced_memory() + tracemalloc.stop() + texts = _field_texts(result) + return { + "elapsed_s": round(elapsed, 4), + "peak_bytes": peak, + "response_bytes": len(texts["response"].encode("utf-8")), + "segments": bucket.as_dict(), + "sha": _field_shas(result), + "_texts": texts, + } + + +def _repeat(birth: dict, body: dict, bucket: _Bucket, times: int) -> list[dict]: + return [_run_once(birth, body, bucket) for _ in range(times)] + + +def _public_row(row: dict) -> dict: + return {key: value for key, value in row.items() if key != "_texts"} + + +def _compare(label: str, left: dict, right: dict, problems: list[str]) -> None: + left_texts = left["_texts"] + right_texts = right["_texts"] + for name in FIELDS: + if left["sha"][name] != right["sha"][name]: + problems.append(f"{label} {name} {_diff_context(left_texts[name], right_texts[name])}") + + +def _freeze_clocks(): + """Pin datetime.now / utcnow in every already imported module. + + ``format=json`` and the reference edition embed wall-clock fields. Freezing + only the engine module leaves other imports on the real clock. + """ + import datetime as dt_mod + + real = dt_mod.datetime + fixed_utc = real(2026, 10, 13, 0, 0, 0) + fixed_local = real(2026, 10, 13, 8, 0, 0) + + class Frozen(real): + @classmethod + def now(cls, tz=None): + if tz is not None: + return fixed_utc.replace(tzinfo=tz) + return fixed_local + + @classmethod + def utcnow(cls): + return fixed_utc + + saved = [] + for module in list(sys.modules.values()): + if module is None: + continue + current = getattr(module, "datetime", None) + if isinstance(current, type) and current.__name__ == "datetime": + saved.append((module, current)) + module.datetime = Frozen + return saved + + +def _restore_clocks(saved) -> None: + for module, current in saved: + module.datetime = current + + +def _strip_generated_at(value): + if isinstance(value, dict): + return {key: _strip_generated_at(item) for key, item in value.items() if key != "generated_at"} + if isinstance(value, list): + return [_strip_generated_at(item) for item in value] + return value + + +def _sha_pair(birth: dict, body: dict, bucket: _Bucket) -> dict: + """Compare one frozen clock on one reused full reading. + + A second ``cmd_full_reading`` appends onto shared trait lists, so two live + engine calls are not a byte test of the render switch. + """ + import copy + + from scripts.jyotish_engine import cmd_full_reading + + _set_flags(False, False) + reading = cmd_full_reading(SimpleNamespace(**birth)) + + class _Reuse: + def _high_rigor_birth_payload(self, _body: dict) -> dict: + return dict(birth) + + def _compute_full_reading_for_thematic(self, _birth: dict): + return copy.deepcopy(reading) + + _set_flags(False, False) + left = _run_once(birth, body, bucket, _Reuse()) + _set_flags(True, False) + right = _run_once(birth, body, bucket, _Reuse()) + problems = [] + _compare("side", left, right, problems) + compared = "full" + if problems: + left_stripped = _sha(_canonical(_strip_generated_at(json.loads(left["_texts"]["response"])))) + right_stripped = _sha(_canonical(_strip_generated_at(json.loads(right["_texts"]["response"])))) + if left_stripped == right_stripped: + problems.clear() + compared = "without_generated_at" + else: + compared = "mismatch" + return { + "compared": compared, + "sha_a": left["sha"]["response"], + "sha_b": right["sha"]["response"], + "problems": problems, + } + + +def _charts() -> dict[str, dict]: + from tests.test_report_english_edition import CASES + + return { + "day": {key: value for key, value in CASES["day"].items() if key != "name"}, + "night": {key: value for key, value in CASES["night"].items() if key != "name"}, + "south": {key: value for key, value in CASES["south"].items() if key != "name"}, + "section2": dict(SECTION2_CHART), + } + + +def _clear_report_cache() -> None: + folder = ROOT / "scratch" / "local" / "report_reference_cache" + if not folder.is_dir(): + return + for path in folder.glob("*.json"): + path.unlink(missing_ok=True) + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--mask-pyjhora", action="store_true") + parser.add_argument("--only-a", action="store_true") + parser.add_argument("--side-only", action="store_true") + args = parser.parse_args() + mask_path = _mask_pyjhora() if args.mask_pyjhora else "" + bucket = _Bucket() + _install_timers(bucket) + os.environ.pop("GITHUB_SHA", None) + report: dict = { + "python": platform.python_version(), + "platform": platform.platform(), + "mask_pyjhora": bool(args.mask_pyjhora), + "mask_path": mask_path, + "only_a": args.only_a, + "charts": {}, + } + problems: list[str] = [] + chart_items = () if args.side_only else _charts().items() + for label, birth in chart_items: + _set_flags(False, False) + arm_a = _repeat(birth, PRODUCTION_BODY, bucket, 3) + entry: dict = {"A": [_public_row(row) for row in arm_a]} + if not args.only_a: + _set_flags(True, False) + arm_b = _repeat(birth, PRODUCTION_BODY, bucket, 3) + for row in arm_b: + _compare(f"{label} B-miss", arm_a[0], row, problems) + os.environ["GITHUB_SHA"] = "report-reference-perf-ab" + _set_flags(True, True) + _clear_report_cache() + miss = _run_once(birth, PRODUCTION_BODY, bucket) + _compare(f"{label} cache-miss", arm_a[0], miss, problems) + hits = _repeat(birth, PRODUCTION_BODY, bucket, 3) + for row in hits: + _compare(f"{label} cache-hit", miss, row, problems) + entry["B_miss"] = [_public_row(row) for row in arm_b] + entry["cache_miss"] = _public_row(miss) + entry["B_hit"] = [_public_row(row) for row in hits] + os.environ.pop("GITHUB_SHA", None) + report["charts"][label] = entry + print(f"{label} done", file=sys.stderr) + if not args.only_a: + saved_clocks = _freeze_clocks() + try: + sample = _charts()["section2"] + report["format_json"] = _sha_pair(sample, {"format": "json", "edition": "full_data"}, bucket) + report["edition_reference"] = _sha_pair( + sample, {"format": "markdown", "edition": "reference"}, bucket, + ) + finally: + _restore_clocks(saved_clocks) + problems.extend(report["format_json"]["problems"]) + problems.extend(report["edition_reference"]["problems"]) + _set_flags(False, False) + from scripts.calculation_profile_contract import hash_bound_result + from scripts.jyotish_engine import build_professional_report_reference_packet, cmd_full_reading + from scripts.professional_report_reference import _export_args + + reading = cmd_full_reading(SimpleNamespace(**sample)) + packet = build_professional_report_reference_packet(reading, _export_args(sample), []) + input_hash = packet["calculation_profile"]["input_hash"] + + def _time_hash(fast: bool) -> float: + _set_flags(fast, False) + started = time.perf_counter() + hash_bound_result(packet, input_hash) + return time.perf_counter() - started + + off = [_time_hash(False) for _ in range(3)] + on = [_time_hash(True) for _ in range(3)] + report["hash_once_s"] = {"off": [round(item, 4) for item in off], "on": [round(item, 4) for item in on]} + report["problems"] = problems + print(json.dumps(report, ensure_ascii=False)) + return 1 if problems else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_report_reference_perf.py b/tests/test_report_reference_perf.py new file mode 100644 index 00000000..752b33a8 --- /dev/null +++ b/tests/test_report_reference_perf.py @@ -0,0 +1,374 @@ +"""Byte-identical report shortcuts and the full_data markdown cache.""" + +from __future__ import annotations + +import copy +import hashlib +import json +import os +import re +import threading +import time +from typing import Any + +import pytest + +from scripts.calculation_profile_contract import hash_bound_result +from scripts.pl9_full_data_export import render_full_data_markdown +from scripts.pl9_language_terms import _thicken_zh_dasha_density, replace_natural_language_term +from scripts.report_perf_flags import ( + REFERENCE_CACHE_DEFAULT, + RENDER_FAST_DEFAULT, + report_reference_cache_enabled, + report_render_fast_enabled, +) +from scripts.professional_report_reference import build_professional_report_reference +from tests.test_report_english_edition import CASES, _packet + + +def _legacy_drop_key_recursive(payload: Any, key_name: str) -> Any: + if isinstance(payload, dict): + return { + key: _legacy_drop_key_recursive(value, key_name) + for key, value in payload.items() + if key != key_name + } + if isinstance(payload, list): + return [_legacy_drop_key_recursive(item, key_name) for item in payload] + return payload + + +def _legacy_drop_dotted_path(payload: Any, path: tuple[str, ...]) -> Any: + if len(path) == 2 and path[0] == "**": + return _legacy_drop_key_recursive(payload, path[1]) + if not path or not isinstance(payload, dict) or path[0] not in payload: + return payload + cloned = dict(payload) + key, *rest = path + if not rest: + cloned.pop(key, None) + return cloned + cloned[key] = _legacy_drop_dotted_path(payload[key], tuple(rest)) + return cloned + + +def _legacy_canonicalize(value: Any) -> Any: + if isinstance(value, dict): + normalized: dict[str, Any] = {} + for key, child in value.items(): + normalized_key = key if isinstance(key, str) else f"__{type(key).__name__}__:{key}" + normalized[normalized_key] = _legacy_canonicalize(child) + return normalized + if isinstance(value, (list, tuple)): + return [_legacy_canonicalize(item) for item in value] + return value + + +def _legacy_payload(result: dict, scope: dict) -> dict: + excluded_top = {"result_hash", "result_binding", "calculation_profile", "calculation_profile_id"} | { + str(key) for key in (scope.get("excluded_top_keys") or ()) + } + payload = {key: value for key, value in result.items() if key not in excluded_top} + for raw_path in scope.get("excluded_paths") or (): + parts = tuple(part for part in str(raw_path).split(".") if part) + if parts: + payload = _legacy_drop_dotted_path(payload, parts) + return payload + + +def _legacy_hash(result: dict, input_hash: str, scope: dict) -> str: + encoded = json.dumps( + _legacy_canonicalize({"input_hash": input_hash, "result": _legacy_payload(result, scope)}), + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + default=str, + ) + return hashlib.sha256(encoded.encode("utf-8")).hexdigest() + + +@pytest.fixture(scope="module") +def day_views(): + """One real day packet. ``delivered`` is the bound object; render uses a copy.""" + from scripts.jyotish_engine import build_professional_report_reference_packet, cmd_full_reading + from scripts.professional_report_reference import _export_args + + os.environ["JYOTISH_REPORT_RENDER_FAST"] = "0" + os.environ["JYOTISH_REPORT_REFERENCE_CACHE"] = "0" + try: + case = CASES["day"] + reading = cmd_full_reading(type("Args", (), dict(case))()) + delivered = build_professional_report_reference_packet(reading, _export_args(dict(case)), []) + packet = dict(delivered) + packet["report_version"] = "pl9_personal_long_report.v3" + zh = render_full_data_markdown(copy.deepcopy(packet)) + english = copy.deepcopy(packet) + english["report_language"] = "en" + en = render_full_data_markdown(english) + finally: + os.environ.pop("JYOTISH_REPORT_RENDER_FAST", None) + os.environ.pop("JYOTISH_REPORT_REFERENCE_CACHE", None) + return delivered, packet, zh, en + + +def test_flags_follow_the_environment_at_call_time(monkeypatch) -> None: + monkeypatch.delenv("JYOTISH_REPORT_RENDER_FAST", raising=False) + monkeypatch.delenv("JYOTISH_REPORT_REFERENCE_CACHE", raising=False) + assert report_render_fast_enabled() is RENDER_FAST_DEFAULT + assert report_reference_cache_enabled() is REFERENCE_CACHE_DEFAULT + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0") + monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "0") + assert report_render_fast_enabled() is False + assert report_reference_cache_enabled() is False + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1") + monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "1") + assert report_render_fast_enabled() is True + assert report_reference_cache_enabled() is True + + +def test_fast_render_matches_the_legacy_bytes(day_views, monkeypatch) -> None: + _delivered, packet, zh_off, en_off = day_views + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1") + monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "0") + zh_on = render_full_data_markdown(copy.deepcopy(packet)) + english = copy.deepcopy(packet) + english["report_language"] = "en" + en_on = render_full_data_markdown(english) + assert zh_on == zh_off + assert en_on == en_off + + +def _both(function, monkeypatch, *args): + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0") + off = function(*args) + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1") + on = function(*args) + assert on == off + return off + + +def test_dasha_density_samples_come_from_the_real_render(day_views, monkeypatch) -> None: + _delivered, _packet_view, zh, _en = day_views + lines = [line for line in zh.splitlines() if line.strip()] + # This fictional chart never emits the density sentence. Fill the existing + # sanitizer sentence with lords that do appear in the render. + lords = [token for token in ("木星", "北交点", "土星", "金星") if token in zh] + assert len(lords) >= 2 + hit = f"{lords[0]}主大运中的{lords[1]}子运，会把{lords[0]}的自然主题带入当前阶段。" + miss = next(line for line in lines if "子运" not in line and "`" not in line) + quoted = next(line for line in lines if "`" in line) + hit_out = _both(_thicken_zh_dasha_density, monkeypatch, hit + "\n") + assert "这一子运不是单独结论" in hit_out or "这个次子运只承担短周期细化" in hit_out + miss_out = _both(_thicken_zh_dasha_density, monkeypatch, miss + "\n") + assert miss_out == miss + quoted_out = _both(_thicken_zh_dasha_density, monkeypatch, quoted + "\n") + assert "`" in quoted_out + assert quoted in quoted_out + + +def test_term_samples_come_from_the_real_render(day_views, monkeypatch) -> None: + _delivered, _packet_view, zh, _en = day_views + lines = zh.splitlines() + raw = "Ashtakavarga" + hit = next(line for line in lines if raw in re.sub(r"`[^`]*`", "", line)) + miss = next(line for line in lines if line.strip() and raw not in line) + span = next(span for span in re.findall(r"`[^`]*`", zh) if raw not in span) + hit_out = _both(replace_natural_language_term, monkeypatch, hit, raw, "探针词") + assert "探针词" in hit_out or hit_out == hit + if re.search(rf"(? None: + packet, _render_packet, _zh, _en = day_views + scope = packet["result_binding"]["binding_scope"] + input_hash = packet["calculation_profile"]["input_hash"] + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1") + assert hash_bound_result(packet, input_hash, scope) == _legacy_hash(packet, input_hash, scope) + assert hash_bound_result(packet, input_hash, scope) == packet["result_hash"] + variant = copy.deepcopy(packet) + variant["probe_layer"] = { + "elapsed_seconds": 1.5, + "called_at": "2001-01-01T00:00:00Z", + "cache_created_at": "created", + "cache_expires_at": "expires", + "nested": {"elapsed_seconds": 9, (1, 2): {"called_at": "inside"}}, + } + summary = variant.setdefault("ai_and_audit", {}).setdefault("summary", {}) + summary["elapsed_seconds"] = 99 + summary["kept"] = "yes" + assert hash_bound_result(variant, input_hash, scope) == _legacy_hash(variant, input_hash, scope) + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0") + assert hash_bound_result(packet, input_hash) == packet["result_hash"] + + +class _CountingHandler: + def __init__(self, birth: dict, packet: dict) -> None: + self.birth = birth + self.packet = packet + self.calls = 0 + self.fail_once = False + self.delay = 0.0 + self.lock = threading.Lock() + + def _high_rigor_birth_payload(self, body: dict) -> dict: + payload = dict(self.birth) + if body.get("today"): + payload["today"] = body["today"] + payload["candidate_range"] = body.get("candidate_range") + return payload + + def _compute_full_reading_for_thematic(self, _birth: dict) -> dict: + with self.lock: + self.calls += 1 + call = self.calls + if self.delay: + time.sleep(self.delay) + if self.fail_once and call == 1: + raise RuntimeError("report_build_failed") + return {"stub": True} + + +def _body(**extra) -> dict: + body = { + "format": "markdown", + "edition": "full_data", + "include_fact_tables": True, + "languages": ["zh", "en"], + "today": "2026-04-07", + } + body.update(extra) + return body + + +@pytest.fixture +def cached_report(monkeypatch, tmp_path, day_views): + import scripts.consult_full_data_cache as cache + import scripts.professional_report_reference as reference + + def _dir(name: str = "consult_full_data_cache"): + path = tmp_path / name + path.mkdir(parents=True, exist_ok=True) + return path + + monkeypatch.setattr(cache, "cache_dir", _dir) + monkeypatch.setattr(reference, "render_full_data_markdown", lambda packet: "en-body" if packet.get("report_language") == "en" else "zh-body") + monkeypatch.setattr(reference, "report_density_packet", lambda _packet: {"facts": 1}) + monkeypatch.setattr(reference, "render_dasha_applicability", lambda _packet, language="zh": f"app-{language}") + monkeypatch.setattr(reference, "clean_reader_appendix_markdown", lambda text: text) + packet = day_views[1] + birth = {key: value for key, value in CASES["day"].items() if key != "name"} + handler = _CountingHandler(birth, packet) + + class _Engine: + def build_professional_report_reference_packet(self, _reading, _args, _packs): + return copy.deepcopy(packet) + + def render_pl9_markdown(self, _packet): + return "reference" + + monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "1") + monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0") + monkeypatch.setenv("GITHUB_SHA", "report-cache-test") + return handler, _Engine(), _dir("report_reference_cache") + + +def _dumps(result: dict) -> str: + return json.dumps(result, ensure_ascii=False, default=str) + + +def test_second_call_reuses_the_cached_response(cached_report) -> None: + handler, engine, folder = cached_report + first = build_professional_report_reference(handler, _body(), engine=engine) + second = build_professional_report_reference(handler, _body(), engine=engine) + assert handler.calls == 1 + assert _dumps(first) == _dumps(second) + assert list(folder.glob("*.json")) + + +@pytest.mark.parametrize("extra", [ + {"today": "2026-05-01"}, + {"languages": ["zh"]}, + {"include_fact_tables": False}, + {"candidate_range": {"kind": "probe"}}, +]) +def test_changed_inputs_miss_the_report_cache(cached_report, extra) -> None: + handler, engine, _folder = cached_report + build_professional_report_reference(handler, _body(), engine=engine) + build_professional_report_reference(handler, _body(**extra), engine=engine) + assert handler.calls == 2 + + +def test_github_sha_is_part_of_the_report_cache_key(cached_report, monkeypatch) -> None: + handler, engine, _folder = cached_report + build_professional_report_reference(handler, _body(), engine=engine) + monkeypatch.setenv("GITHUB_SHA", "report-cache-other") + build_professional_report_reference(handler, _body(), engine=engine) + assert handler.calls == 2 + + +def test_json_format_and_disabled_cache_write_nothing(cached_report, monkeypatch) -> None: + handler, engine, folder = cached_report + for path in folder.glob("*.json"): + path.unlink() + build_professional_report_reference( + handler, + {"format": "json", "edition": "full_data", "today": "2026-04-07"}, + engine=engine, + ) + assert list(folder.glob("*.json")) == [] + monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "0") + build_professional_report_reference(handler, _body(), engine=engine) + assert list(folder.glob("*.json")) == [] + monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "1") + monkeypatch.setenv("GITHUB_SHA", "unknown") + build_professional_report_reference(handler, _body(), engine=engine) + assert list(folder.glob("*.json")) == [] + + +def test_concurrent_same_key_builds_once(cached_report) -> None: + handler, engine, _folder = cached_report + handler.delay = 0.2 + barrier = threading.Barrier(2) + errors = [] + + def run() -> None: + try: + barrier.wait() + build_professional_report_reference(handler, _body(), engine=engine) + except Exception as exc: # pragma: no cover - recorded for the assertion + errors.append(exc) + + threads = [threading.Thread(target=run) for _ in range(2)] + for thread in threads: + thread.start() + for thread in threads: + thread.join() + assert errors == [] + assert handler.calls == 1 + + +def test_failed_build_is_not_cached(cached_report) -> None: + handler, engine, folder = cached_report + handler.fail_once = True + with pytest.raises(RuntimeError): + build_professional_report_reference(handler, _body(), engine=engine) + assert list(folder.glob("*.json")) == [] + build_professional_report_reference(handler, _body(), engine=engine) + assert handler.calls == 2 + build_professional_report_reference(handler, _body(), engine=engine) + assert handler.calls == 2 + + +def test_report_cache_directory_prunes_old_files(cached_report) -> None: + handler, engine, folder = cached_report + stale = folder / "stale-probe.json" + stale.write_text("{}", encoding="utf-8") + old = time.time() - (3 * 24 * 60 * 60) + os.utime(stale, (old, old)) + build_professional_report_reference(handler, _body(), engine=engine) + assert not stale.exists() From f122f8b7d7ca369328ce9e1160805202d104d1ac Mon Sep 17 00:00:00 2001 From: Jesse_Chen Date: Wed, 14 Oct 2026 11:02:37 +0800 Subject: [PATCH 2/2] docs(tasks): frontend dead code fix brief (agent loop, issue #63, bounce 1) Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01LutgzyJqNhPpZy1nFsJtN3 --- docs/tasks/README.md | 1 + .../TASK-frontend-dead-code-fix-20261014.md | 33 +++++++++++++++++++ 2 files changed, 34 insertions(+) create mode 100644 docs/tasks/TASK-frontend-dead-code-fix-20261014.md diff --git a/docs/tasks/README.md b/docs/tasks/README.md index 6967060a..c49ff179 100644 --- a/docs/tasks/README.md +++ b/docs/tasks/README.md @@ -478,6 +478,7 @@ | `TASK-consult-prep-parallel-20261013.md` | — | 普通对话发问前准备步骤并行、模型版本短缓存、多领域并发开关（BUG-1337～1339，A/B） | 待领取 | `codex/consult-prep-parallel-20261013` | | `TASK-consult-prep-parallel-fix-20261014.md` | — | **#62 第 1 次打回**：`a0e223e2` 业务逻辑通过；两条源码文本断言未跟改（§7.8）、A/B 脚本从 `frontend/tests` 导入（BUG-1140） | 待领取 | `codex/consult-prep-parallel-20261013` | | `TASK-frontend-dead-code-20261013.md` | — | 前端死代码 17 文件与无调用方接口删除，含未登录重计算接口（BUG-1342） | 待领取 | `codex/frontend-dead-code-20261013` | +| `TASK-frontend-dead-code-fix-20261014.md` | — | **#63 第 1 次打回**：`1f9947a0` 其余全过；类名白名单留了已删组件的 `personal-report-additional-charts` | 待领取 | `codex/frontend-dead-code-20261013` | | `TASK-api-chart-subprocess-gate-20261013.md` | — | 合盘不带 skip 串行排盘、报告无效子进程、名额轻重混用（只模拟）（BUG-1333～1334，A/B） | 待领取（排队：api-disk-cache-retention + report-reference-perf） | `codex/api-chart-subprocess-gate-20261013` | | `TASK-answer-effort-ab-20261013.md` | — | 答题步推理强度开关 + A/B（需模型 key） | 待领取（排队：consult-prep-parallel） | `codex/answer-effort-ab-20261013` | | `TASK-consult-route-split-20261013.md` | — | consult POST 拆分第一阶段：开流前加载 + 计费结算 | 待领取（排队：consult-prep-parallel、answer-effort-ab） | `codex/consult-route-split-20261013` | diff --git a/docs/tasks/TASK-frontend-dead-code-fix-20261014.md b/docs/tasks/TASK-frontend-dead-code-fix-20261014.md new file mode 100644 index 00000000..47de3f12 --- /dev/null +++ b/docs/tasks/TASK-frontend-dead-code-fix-20261014.md @@ -0,0 +1,33 @@ +# TASK · 前端死代码删除修复单（issue #63，第 1 次打回） + +- 日期：2026-10-14 +- 原任务书：`docs/tasks/TASK-frontend-dead-code-20261013.md` +- 被验 SHA：`1f9947a0d733e34b7754cce058e20b023791fdd8`（`codex/frontend-dead-code-20261013`） +- 执行方：Grok，同一分支续做（先 merge 最新 `origin/staging`，不改写已推历史） +- BUG 编号：沿用 BUG-1342 + +## 1. 验收结论：一条未通过，其余全部通过 + +| 检查 | 结果 | 证据 | +| --- | --- | --- | +| 删除范围 | 通过 | 删掉的文件与三个路由都在原任务书清单内；全仓 grep 三个路由路径、被删模块名，`frontend/src`、`scripts`、`deploy`、`.gitea` 无残留引用 | +| 改既有断言 | 通过 | 每处三栏齐全；否定合同（不得挂内部徽章 / 内部面板）保留；`/api/birth-rectification` 改成「路由不存在且无代理」 | +| tsc / lint / next build | 通过 | 0 错 / 0 error（warning 125→123）/ `/` 仍 Static；首屏 gzip −312 字节 | +| Python 改动的两份测试 | 通过 | — | +| 前端全量 `npm test`（与 staging 逐名比对） | **未通过：新增 1 条** | `frontend/tests/class-name-definition-contract.test.ts`「the knownUnstyled allowlist has no stale entries」：`actual: ['personal-report-additional-charts']` | + +**根因**：`class-name-definition-contract.test.ts` 第 62 行的 `knownUnstyled` 白名单里有 `personal-report-additional-charts`，这个类名唯一的使用者是已删除的 `personal-report-document-view.tsx`。合同要求白名单里不能有已经没人用的条目。 + +## 2. 修法要求 + +1. 从 `knownUnstyled` 删掉 `"personal-report-additional-charts"` 这一项，旁边按 §7.3 写三栏（原值：白名单含该项；新值：删除；原因：唯一使用者随 BUG-1342 删除，白名单不得留陈旧条目）。 +2. 再查一遍：被删文件里用到的其它类名，是否还出现在别的白名单 / 合同里（`grep -rn "knownUnstyled\|allowlist" frontend/tests` 后逐个核对）。 +3. 业务代码不改。PROGRESS 末尾记一行。 + +## 3. 验收标准 + +前端全量 `npm test` 失败清单与 staging 基线逐名相同（本机 Docker 类失败除外）。 + +## 4. 让步顺序 + +无。第 1 次打回。