merge origin/staging into consult prep parallel (issue #62)
Docs-only catch-up; agent-loop rule A. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LutgzyJqNhPpZy1nFsJtN3
This commit is contained in:
@@ -1,5 +1,11 @@
|
||||
# 印度占星 Skill 更新日志
|
||||
|
||||
## 2026-10-14 — 个人长报告引擎出稿提速(同盘同日重试秒回)
|
||||
|
||||
- 同一张盘、同一天、同一个已发布版本,再次生成完整数据版长报告时直接用刚才的结果,本机大约 0.06 秒。第一次生成仍走原来的计算。版本号缺失时不缓存。
|
||||
- 中文替换和溯源哈希的快捷算法已留下,但本机只快了大约两成,没有达到开通门槛,默认仍走原算法。
|
||||
- Skill 版本不变(BUG-1335、BUG-1336)。
|
||||
|
||||
## 2026-10-14 — 服务器不再长期保留出生资料副本
|
||||
|
||||
- 排盘缓存、官方快照和相关临时文件会按期限删掉,不再一直堆在服务器上。审计用的快照文件默认不再落地。失败的官方快照只短时间复用。
|
||||
|
||||
+18052
-18022
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,44 @@
|
||||
# PROGRESS · 长报告出稿提速(2026-10-14)
|
||||
|
||||
- 分支:`codex/report-reference-perf-20261013`
|
||||
- 实际基线:`origin/staging` `66cd60081f37a57680274633fcb3fb942d01bb2f`(任务书写的 `77057a819`、计划评论里的 `2994588ad` 都更早,按当前 staging 开工)
|
||||
- `scripts/jyotish_api_server.py`:10335 行,零改动
|
||||
- 历史最大号在本基线上是 BUG-1322。BUG-1323~1334 已被其它未合入任务书占用。本单用任务书指定的 BUG-1335、BUG-1336。
|
||||
|
||||
## 做了什么
|
||||
|
||||
1. 两个开关在调用时读环境变量。渲染快捷路径默认关,同盘缓存默认开。
|
||||
2. 中文和英文的大运密度补句:正则预编译,行内先看必出现的字面量,没有才不搜索;向前看的五行只在可能命中时才拼。术语替换:正文里没有这个词就原样返回,编译结果按词缓存。
|
||||
3. 溯源哈希在开关打开时,一次遍历同时删掉 `**.键` 并做键名规范化。覆盖范围和排除集没变。没有跳过中间那次绑定,因为那会改 `format=json` 的键顺序。
|
||||
4. 咨询卡缓存的单飞和两天清理多了一个子目录参数,咨询卡自己的键和目录不变。长报告只在完整数据版 + markdown、开关开、并且 `GITHUB_SHA` 是具体版本号时,才把整份响应放进 `scratch/local/report_reference_cache/`。查缓存发生在重计算之前。命中打一行不含资料的日志。
|
||||
|
||||
## 自测
|
||||
|
||||
| 检查 | 命令 | 结果 |
|
||||
| --- | --- | --- |
|
||||
| 任务书列出的 pytest | `.venv\Scripts\python.exe -m pytest` 后接任务书第 8 节那一串文件 `-q` | 退出码 0。缓存默认改成开之后又跑了一遍,仍是退出码 0 |
|
||||
| 屏蔽 PyJHora 的 A/B | `python scripts/research/report_reference_perf_ab.py --mask-pyjhora` | 四张盘五字段 sha 相同;端到端下降 20%~23%,未达 35%;缓存命中中位约 0.06 秒 |
|
||||
| 同一份解盘的 json / reference | 同上脚本 `--side-only` | 退出码 0。时钟钉死且复用同一次解盘后,两版 sha 相同 |
|
||||
| 不屏蔽 | 未跑 | 本机没有 PyJHora,和屏蔽走同一条导入失败路径 |
|
||||
|
||||
数字见 `docs/testing/report-reference-perf-ab-20261013.md`。
|
||||
|
||||
## 改既有断言
|
||||
|
||||
无。原值 / 新值 / 原因:没有改旧断言。
|
||||
|
||||
## 未做
|
||||
|
||||
- 整篇替换逐条预检(让步顺序里最先砍的一项)。中文渲染只降了 20%~28%,不到 40%,没有为了过线改输出。
|
||||
- 跳过中间绑定。
|
||||
- 关掉没人读的质量门。任务书写明这要另开单,本单不动。
|
||||
- 旧路径退役。开关还在。
|
||||
|
||||
## 默认值
|
||||
|
||||
- `JYOTISH_REPORT_RENDER_FAST` 默认关。字节一致,内存还降了约 12%,但端到端没到 35%。
|
||||
- `JYOTISH_REPORT_REFERENCE_CACHE` 默认开。命中与未命中字节一致,命中中位约 0.06 秒。版本号缺失或是 `unknown` 时不缓存。
|
||||
|
||||
## 第 1 次打回
|
||||
|
||||
合并最新 `origin/staging` 后,把 `report_reference_cache` 登记进 `OFFLINE_SCRATCH_NAMES`。清理逻辑没改:这个目录仍由 `consult_full_data_cache.get_or_build` 在每次调用时删掉两天前的文件。没有改两个缓存模块的行为。
|
||||
@@ -473,11 +473,12 @@
|
||||
| `TASK-home-sidebar-rerender-20261013.md` | `PROGRESS-home-sidebar-rerender-20261013.md` | 流式回答时侧栏每帧整栏重渲染(BUG-1329,A/B) | 待验收 | `codex/home-sidebar-rerender-20261013` |
|
||||
| `TASK-home-sidebar-rerender-fix-20261014.md` | `PROGRESS-home-sidebar-rerender-20261013.md` | **#59 第 1 次打回**:`63509431` 其余全过;`chat-navigation-a11y-contract` 按源码文本断言 `onOpenBilling` 旧写法未跟改(§7.8) | 待验收 | `codex/home-sidebar-rerender-20261013` |
|
||||
| `TASK-api-disk-cache-retention-20261013.md` | `PROGRESS-api-disk-cache-retention-20261013.md` | API 磁盘缓存只进不出、VedAstro 审计文件留存出生资料、失败快照久缓存、缓存键规范化(BUG-1330~1332,A/B) | 待验收 | `codex/api-disk-cache-retention-20261013` |
|
||||
| `TASK-report-reference-perf-20261013.md` | — | 长报告出稿提速:术语替换命中预检、哈希单次遍历、同盘复用(BUG-1335~1336,A/B) | 待领取 | `codex/report-reference-perf-20261013` |
|
||||
| `TASK-report-reference-perf-fix-20261014.md` | — | **#61 第 1 次打回**:`69c739fb` 本单全过(Linux 复跑逐字节一致、命中 0.03 s);与 #60 合并后新缓存目录未登记进 `api_scratch_retention`,合同测试红 | 待领取 | `codex/report-reference-perf-20261013` |
|
||||
| `TASK-report-reference-perf-20261013.md` | `PROGRESS-report-reference-perf-20261013.md` | 长报告出稿提速:术语替换命中预检、哈希单次遍历、同盘复用(BUG-1335~1336,A/B) | 待验收 | `codex/report-reference-perf-20261013` |
|
||||
| `TASK-report-reference-perf-fix-20261014.md` | `PROGRESS-report-reference-perf-20261013.md` | **#61 第 1 次打回**:`69c739fb` 本单全过;与清理规则合并后,长报告缓存目录补进自行清理名单 | 待验收 | `codex/report-reference-perf-20261013` |
|
||||
| `TASK-consult-prep-parallel-20261013.md` | `PROGRESS-consult-prep-parallel-20261013.md` | 普通对话发问前准备步骤并行、模型版本短缓存、多领域并发开关(BUG-1337~1339,A/B) | 待验收 | `codex/consult-prep-parallel-20261013` |
|
||||
| `TASK-consult-prep-parallel-fix-20261014.md` | `PROGRESS-consult-prep-parallel-20261013.md` | **#62 第 1 次打回**:两条源码文本检查改认新位置,对照脚本挪进测试目录 | 待验收 | `codex/consult-prep-parallel-20261013` |
|
||||
| `TASK-frontend-dead-code-20261013.md` | — | 前端死代码 17 文件与无调用方接口删除,含未登录重计算接口(BUG-1342) | 待领取 | `codex/frontend-dead-code-20261013` |
|
||||
| `TASK-frontend-dead-code-fix-20261014.md` | — | **#63 第 1 次打回**:`1f9947a0` 其余全过;类名白名单留了已删组件的 `personal-report-additional-charts` | 待领取 | `codex/frontend-dead-code-20261013` |
|
||||
| `TASK-api-chart-subprocess-gate-20261013.md` | — | 合盘不带 skip 串行排盘、报告无效子进程、名额轻重混用(只模拟)(BUG-1333~1334,A/B) | 待领取(排队:api-disk-cache-retention + report-reference-perf) | `codex/api-chart-subprocess-gate-20261013` |
|
||||
| `TASK-answer-effort-ab-20261013.md` | — | 答题步推理强度开关 + A/B(需模型 key) | 待领取(排队:consult-prep-parallel) | `codex/answer-effort-ab-20261013` |
|
||||
| `TASK-consult-route-split-20261013.md` | — | consult POST 拆分第一阶段:开流前加载 + 计费结算 | 待领取(排队:consult-prep-parallel、answer-effort-ab) | `codex/consult-route-split-20261013` |
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
# TASK · 前端死代码删除修复单(issue #63,第 1 次打回)
|
||||
|
||||
- 日期:2026-10-14
|
||||
- 原任务书:`docs/tasks/TASK-frontend-dead-code-20261013.md`
|
||||
- 被验 SHA:`1f9947a0d733e34b7754cce058e20b023791fdd8`(`codex/frontend-dead-code-20261013`)
|
||||
- 执行方:Grok,同一分支续做(先 merge 最新 `origin/staging`,不改写已推历史)
|
||||
- BUG 编号:沿用 BUG-1342
|
||||
|
||||
## 1. 验收结论:一条未通过,其余全部通过
|
||||
|
||||
| 检查 | 结果 | 证据 |
|
||||
| --- | --- | --- |
|
||||
| 删除范围 | 通过 | 删掉的文件与三个路由都在原任务书清单内;全仓 grep 三个路由路径、被删模块名,`frontend/src`、`scripts`、`deploy`、`.gitea` 无残留引用 |
|
||||
| 改既有断言 | 通过 | 每处三栏齐全;否定合同(不得挂内部徽章 / 内部面板)保留;`/api/birth-rectification` 改成「路由不存在且无代理」 |
|
||||
| tsc / lint / next build | 通过 | 0 错 / 0 error(warning 125→123)/ `/` 仍 Static;首屏 gzip −312 字节 |
|
||||
| Python 改动的两份测试 | 通过 | — |
|
||||
| 前端全量 `npm test`(与 staging 逐名比对) | **未通过:新增 1 条** | `frontend/tests/class-name-definition-contract.test.ts`「the knownUnstyled allowlist has no stale entries」:`actual: ['personal-report-additional-charts']` |
|
||||
|
||||
**根因**:`class-name-definition-contract.test.ts` 第 62 行的 `knownUnstyled` 白名单里有 `personal-report-additional-charts`,这个类名唯一的使用者是已删除的 `personal-report-document-view.tsx`。合同要求白名单里不能有已经没人用的条目。
|
||||
|
||||
## 2. 修法要求
|
||||
|
||||
1. 从 `knownUnstyled` 删掉 `"personal-report-additional-charts"` 这一项,旁边按 §7.3 写三栏(原值:白名单含该项;新值:删除;原因:唯一使用者随 BUG-1342 删除,白名单不得留陈旧条目)。
|
||||
2. 再查一遍:被删文件里用到的其它类名,是否还出现在别的白名单 / 合同里(`grep -rn "knownUnstyled\|allowlist" frontend/tests` 后逐个核对)。
|
||||
3. 业务代码不改。PROGRESS 末尾记一行。
|
||||
|
||||
## 3. 验收标准
|
||||
|
||||
前端全量 `npm test` 失败清单与 staging 基线逐名相同(本机 Docker 类失败除外)。
|
||||
|
||||
## 4. 让步顺序
|
||||
|
||||
无。第 1 次打回。
|
||||
@@ -0,0 +1,61 @@
|
||||
# 长报告出稿 A/B(2026-10-14)
|
||||
|
||||
Windows 本机,Python 3.11.7,`Windows-10-10.0.26200-SP0`。屏蔽 PyJHora(生产镜像不装这个库)。本机本身也没有 PyJHora,所以没有再跑一轮不屏蔽的重复计时。计时开了 tracemalloc,绝对秒数比任务书里的 Linux 基线高,只比较同一进程里的 A 和 B。
|
||||
|
||||
A = 两个开关关。B = 两个开关开。四张虚构盘,每组 3 次。生产请求是完整数据版、markdown、中英都要、带事实表。没有模型钥匙。
|
||||
|
||||
## 端到端(未命中,秒)
|
||||
|
||||
| 盘 | A 三次 | A 中位 | B 三次 | B 中位 | 下降 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| day | 71.31 / 64.03 / 64.01 | 64.03 | 51.05 / 50.77 / 51.74 | 51.05 | 20.3% |
|
||||
| night | 66.25 / 66.11 / 66.66 | 66.25 | 53.32 / 52.57 / 50.43 | 52.57 | 20.7% |
|
||||
| south | 66.03 / 66.00 / 62.69 | 66.00 | 49.97 / 52.79 / 52.68 | 52.68 | 20.2% |
|
||||
| section2 | 71.01 / 108.37 / 74.19 | 74.19 | 57.42 / 57.12 / 71.80 | 57.42 | 22.6% |
|
||||
|
||||
门槛是未命中端到端下降至少 35%。四张盘都没到。渲染快捷路径默认关闭。
|
||||
|
||||
## 分段中位数(秒)
|
||||
|
||||
| 盘 | 中文渲染 A → B | 英文渲染 A → B | 建包 A → B |
|
||||
| --- | --- | --- | --- |
|
||||
| day | 23.55 → 17.04(−27.6%) | 8.27 → 8.67 | 22.55 → 16.08 |
|
||||
| night | 23.42 → 17.19(−26.6%) | 8.75 → 8.41 | 23.70 → 17.21 |
|
||||
| south | 23.89 → 17.39(−27.2%) | 8.66 → 8.74 | 22.90 → 16.87 |
|
||||
| section2 | 22.26 → 17.80(−20.0%) | 8.14 → 8.54 | 31.24 → 21.08 |
|
||||
|
||||
中文渲染门槛是下降至少 40%。没到。没有继续加整篇替换的预检。
|
||||
|
||||
## 内存与字节
|
||||
|
||||
tracemalloc 峰值:B 约为 A 的 0.87 到 0.88,没有升高。
|
||||
|
||||
五份输出(整份响应、中文、英文、事实表、适用性)A、B 未命中、缓存命中的 sha256 全部相同。
|
||||
|
||||
| 盘 | 响应字节 | 响应 sha256 前 12 位 |
|
||||
| --- | --- | --- |
|
||||
| day | 2289867 | 5b542359d606 |
|
||||
| night | 2321638 | 89baa960c054 |
|
||||
| south | 2314369 | 250bf1ab246e |
|
||||
| section2 | 2238092 | 1d7afd481b1a |
|
||||
|
||||
## 同盘缓存(命中,秒)
|
||||
|
||||
| 盘 | 三次 | 中位 |
|
||||
| --- | --- | --- |
|
||||
| day | 0.059 / 0.054 / 0.057 | 0.057 |
|
||||
| night | 0.057 / 0.054 / 0.058 | 0.057 |
|
||||
| south | 0.059 / 0.054 / 0.057 | 0.057 |
|
||||
| section2 | 0.063 / 0.066 / 0.057 | 0.063 |
|
||||
|
||||
命中字节与未命中相同,中位数远低于 1.5 秒。缓存开关默认开。`GITHUB_SHA` 缺失或为 `unknown` 时仍然关。
|
||||
|
||||
## 单次溯源哈希(交付包,秒)
|
||||
|
||||
关:1.132 / 1.099 / 1.115,中位 1.115。开:0.402 / 0.466 / 0.388,中位 0.402。下降 64%,过了 50% 那一档。它跟渲染快捷路径用同一个开关,所以默认也是关。
|
||||
|
||||
## json 与 reference 版
|
||||
|
||||
同一份解盘、时钟钉死后,`format=json` 和 reference 版 markdown 在快捷路径关与开时 sha256 相同(`aa1352619b7a`、`cbadf95f549a`)。对比的是整份响应,没有丢掉字段。
|
||||
|
||||
连续两次重新解盘会对不齐:有的特质列表挂在类上,第二次会再追加。那不是这条快捷路径改出来的。生产要的完整数据 markdown 在多次重算里仍然逐字节相同。
|
||||
@@ -42,6 +42,8 @@ REGISTERED_DIR_NAMES = frozenset(
|
||||
OFFLINE_SCRATCH_NAMES = frozenset(
|
||||
{
|
||||
"consult_full_data_cache",
|
||||
# 与 consult_full_data_cache 同走 get_or_build,每次调用先 prune_cache(name=…) 删两天前的文件,自己会清;含出生资料,保留期 2 天。
|
||||
"report_reference_cache",
|
||||
"ocr_extract",
|
||||
"three_engine_parity",
|
||||
"public_release_deny_patterns.txt",
|
||||
|
||||
@@ -486,15 +486,26 @@ def _drop_dotted_path(payload: Any, path: tuple[str, ...]) -> Any:
|
||||
return cloned
|
||||
|
||||
|
||||
def result_payload_for_binding(
|
||||
def _render_fast() -> bool:
|
||||
try:
|
||||
from scripts.report_perf_flags import report_render_fast_enabled
|
||||
except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution
|
||||
from report_perf_flags import report_render_fast_enabled
|
||||
return report_render_fast_enabled()
|
||||
|
||||
|
||||
def _excluded_binding_top(binding_scope: dict[str, Any]) -> set[str]:
|
||||
return RESULT_BINDING_ENVELOPE_KEYS | {
|
||||
str(key) for key in (binding_scope.get("excluded_top_keys") or ())
|
||||
}
|
||||
|
||||
|
||||
def _result_payload_for_binding_legacy(
|
||||
result: dict[str, Any],
|
||||
scope: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Business payload that participates in result_hash."""
|
||||
binding_scope = scope if isinstance(scope, dict) else default_result_binding_scope()
|
||||
excluded_top = RESULT_BINDING_ENVELOPE_KEYS | {
|
||||
str(key) for key in (binding_scope.get("excluded_top_keys") or ())
|
||||
}
|
||||
excluded_top = _excluded_binding_top(binding_scope)
|
||||
payload = {key: value for key, value in result.items() if key not in excluded_top}
|
||||
for raw_path in binding_scope.get("excluded_paths") or ():
|
||||
parts = tuple(part for part in str(raw_path).split(".") if part)
|
||||
@@ -503,17 +514,73 @@ def result_payload_for_binding(
|
||||
return payload
|
||||
|
||||
|
||||
def _drop_star_keys_and_canonicalize(payload: Any, star_keys: frozenset[str]) -> Any:
|
||||
"""Drop ``**.key`` names and canonicalize keys in one walk.
|
||||
|
||||
String keys use the same inequality as ``_drop_key_recursive``. Non-string
|
||||
keys are kept, then renamed with the same rule as ``canonicalize_result_payload``.
|
||||
"""
|
||||
if isinstance(payload, dict):
|
||||
normalized: dict[str, Any] = {}
|
||||
for key, child in payload.items():
|
||||
if isinstance(key, str):
|
||||
if key in star_keys:
|
||||
continue
|
||||
normalized_key = key
|
||||
else:
|
||||
normalized_key = f"__{type(key).__name__}__:{key}"
|
||||
normalized[normalized_key] = _drop_star_keys_and_canonicalize(child, star_keys)
|
||||
return normalized
|
||||
if isinstance(payload, (list, tuple)):
|
||||
return [_drop_star_keys_and_canonicalize(item, star_keys) for item in payload]
|
||||
return payload
|
||||
|
||||
|
||||
def _result_payload_for_binding_fast(
|
||||
result: dict[str, Any],
|
||||
scope: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
binding_scope = scope if isinstance(scope, dict) else default_result_binding_scope()
|
||||
excluded_top = _excluded_binding_top(binding_scope)
|
||||
payload = {key: value for key, value in result.items() if key not in excluded_top}
|
||||
star_keys: set[str] = set()
|
||||
for raw_path in binding_scope.get("excluded_paths") or ():
|
||||
parts = tuple(part for part in str(raw_path).split(".") if part)
|
||||
if len(parts) == 2 and parts[0] == "**":
|
||||
star_keys.add(parts[1])
|
||||
continue
|
||||
if parts:
|
||||
payload = _drop_dotted_path(payload, parts)
|
||||
return _drop_star_keys_and_canonicalize(payload, frozenset(star_keys))
|
||||
|
||||
|
||||
def result_payload_for_binding(
|
||||
result: dict[str, Any],
|
||||
scope: dict[str, Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Business payload that participates in result_hash."""
|
||||
if _render_fast():
|
||||
return _result_payload_for_binding_fast(result, scope)
|
||||
return _result_payload_for_binding_legacy(result, scope)
|
||||
|
||||
|
||||
def hash_bound_result(
|
||||
result: dict[str, Any],
|
||||
input_hash: str,
|
||||
scope: dict[str, Any] | None = None,
|
||||
) -> str:
|
||||
"""SHA-256 of the canonical bound payload. Does not mutate ``result``."""
|
||||
encoded = json.dumps(
|
||||
canonicalize_result_payload({
|
||||
payload = result_payload_for_binding(result, scope)
|
||||
if _render_fast():
|
||||
# The fast payload is already canonical, including the star-key drop.
|
||||
canonical = {"input_hash": input_hash, "result": payload}
|
||||
else:
|
||||
canonical = canonicalize_result_payload({
|
||||
"input_hash": input_hash,
|
||||
"result": result_payload_for_binding(result, scope),
|
||||
}),
|
||||
"result": payload,
|
||||
})
|
||||
encoded = json.dumps(
|
||||
canonical,
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
|
||||
@@ -57,8 +57,8 @@ def _release_warm_slot() -> None:
|
||||
_warm_inflight -= 1
|
||||
|
||||
|
||||
def cache_dir() -> Path:
|
||||
path = ROOT / "scratch" / "local" / "consult_full_data_cache"
|
||||
def cache_dir(name: str = "consult_full_data_cache") -> Path:
|
||||
path = ROOT / "scratch" / "local" / name
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
@@ -118,15 +118,15 @@ def cache_key(identity: dict[str, Any]) -> str:
|
||||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _cache_path(identity: dict[str, Any]) -> Path:
|
||||
return cache_dir() / f"{cache_key(identity)}.json"
|
||||
def _cache_path(identity: dict[str, Any], name: str = "consult_full_data_cache") -> Path:
|
||||
return cache_dir(name) / f"{cache_key(identity)}.json"
|
||||
|
||||
|
||||
def prune_cache(now: float | None = None) -> int:
|
||||
def prune_cache(now: float | None = None, *, name: str = "consult_full_data_cache") -> int:
|
||||
"""Drop cache files last written more than two days ago."""
|
||||
cutoff = (time.time() if now is None else now) - KEEP_SECONDS
|
||||
removed = 0
|
||||
folder = cache_dir()
|
||||
folder = cache_dir(name)
|
||||
for path in folder.glob("*.json"):
|
||||
try:
|
||||
if path.stat().st_mtime < cutoff:
|
||||
@@ -147,15 +147,21 @@ def get_or_build(
|
||||
builder: PacketBuilder | None = None,
|
||||
*,
|
||||
clock: Callable[[], float] | None = None,
|
||||
cache_name: str = "consult_full_data_cache",
|
||||
json_default: Callable[..., Any] | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Return the packet. Concurrent callers for one key share one build."""
|
||||
prune_cache()
|
||||
"""Return the packet. Concurrent callers for one key share one build.
|
||||
|
||||
``cache_name`` selects the scratch subdirectory. The consult card keeps the
|
||||
default. A non-default name does not change the consult key or its files.
|
||||
"""
|
||||
prune_cache(name=cache_name)
|
||||
now = clock or time.perf_counter
|
||||
started = now()
|
||||
path = _cache_path(identity)
|
||||
path = _cache_path(identity, cache_name)
|
||||
if path.is_file():
|
||||
return _read_cached(path, started, now)
|
||||
key = cache_key(identity)
|
||||
key = f"{cache_name}\n{cache_key(identity)}"
|
||||
with _flights_guard:
|
||||
flight = _flights.get(key)
|
||||
owner = flight is None
|
||||
@@ -177,7 +183,10 @@ def get_or_build(
|
||||
builder = build_full_data_packet
|
||||
packet = builder(identity)
|
||||
temporary = path.with_suffix(".json.tmp")
|
||||
temporary.write_text(json.dumps(packet, ensure_ascii=False, separators=(",", ":")), encoding="utf-8")
|
||||
dump_kwargs: dict[str, Any] = {"ensure_ascii": False, "separators": (",", ":")}
|
||||
if json_default is not None:
|
||||
dump_kwargs["default"] = json_default
|
||||
temporary.write_text(json.dumps(packet, **dump_kwargs), encoding="utf-8")
|
||||
temporary.replace(path)
|
||||
elapsed = now() - started
|
||||
return {
|
||||
|
||||
@@ -7,9 +7,15 @@ It only changes reader-facing labels; it does not alter calculation payloads.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import functools
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
try:
|
||||
from scripts.report_perf_flags import report_render_fast_enabled
|
||||
except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution
|
||||
from report_perf_flags import report_render_fast_enabled
|
||||
|
||||
|
||||
# Bilingual planet tokens from yoga combination text, longest first.
|
||||
# Dignity is glued on with no space ("木星Jupiterexalted"), so the long form
|
||||
@@ -131,6 +137,11 @@ def translate_zh_glued_phrases(text: str) -> str:
|
||||
return text
|
||||
|
||||
|
||||
@functools.lru_cache(maxsize=512)
|
||||
def _natural_language_term_pattern(raw: str, plural: str) -> re.Pattern:
|
||||
return re.compile(rf"(?<![A-Za-z0-9_]){re.escape(raw)}{plural}(?![A-Za-z0-9_])")
|
||||
|
||||
|
||||
def replace_natural_language_term(text: str, raw: str, public: str) -> str:
|
||||
"""Replace one glossary term in prose only.
|
||||
|
||||
@@ -141,7 +152,14 @@ def replace_natural_language_term(text: str, raw: str, public: str) -> str:
|
||||
if not text or not raw:
|
||||
return text
|
||||
plural = "s?" if re.search(r"[A-Za-z]$", raw) and not raw.endswith("s") else ""
|
||||
pattern = re.compile(rf"(?<![A-Za-z0-9_]){re.escape(raw)}{plural}(?![A-Za-z0-9_])")
|
||||
if report_render_fast_enabled():
|
||||
# Case-sensitive literal. If it is absent, split-and-join would return
|
||||
# the same bytes, so the compile and the scan can be skipped.
|
||||
if raw not in text:
|
||||
return text
|
||||
pattern = _natural_language_term_pattern(raw, plural)
|
||||
else:
|
||||
pattern = re.compile(rf"(?<![A-Za-z0-9_]){re.escape(raw)}{plural}(?![A-Za-z0-9_])")
|
||||
parts = re.split(r"(`[^`]*`)", text)
|
||||
return "".join(
|
||||
part if index % 2 else pattern.sub(public, part)
|
||||
@@ -555,12 +573,98 @@ def _restore_zh_nakshatra_glosses(markdown: str) -> str:
|
||||
)
|
||||
|
||||
|
||||
# Every AD/PD hit contains these literals. Patterns stay case-sensitive.
|
||||
_ZH_AD_DENSITY_RE = re.compile(
|
||||
r"(.+?)主大运中的(.+?)子运,会把(.+?)的自然主题带入当前阶段。"
|
||||
)
|
||||
_ZH_PD_DENSITY_RE = re.compile(
|
||||
r"(.+?)子运中的(.+?)次子运,会在(.+?)背景下呈现(.+?)的短周期结果。"
|
||||
)
|
||||
_ZH_AD_ANCHORS = ("主大运中的", "子运,会把", "的自然主题带入当前阶段。")
|
||||
_ZH_PD_ANCHORS = ("次子运,会在", "背景下呈现", "的短周期结果。")
|
||||
_EN_MD_HEADING_RE = re.compile(r"^#{2,4}\s+(.+?\bMaha Dasha\b.*)$")
|
||||
_EN_AD_DENSITY_RE = re.compile(
|
||||
r"The Antar Dasha of (.+?) activates (.+?)'s natural themes inside the broader Maha Dasha of (.+?)\."
|
||||
)
|
||||
_EN_PD_DENSITY_RE = re.compile(
|
||||
r"In the Pratyantar Dasha of (.+?) in the Antar Dasha of (.+?), (.+?)'s significations give the short-period result inside the (.+?) background\."
|
||||
)
|
||||
_EN_AD_ANCHORS = (
|
||||
"The Antar Dasha of ",
|
||||
" activates ",
|
||||
"'s natural themes inside the broader Maha Dasha of ",
|
||||
)
|
||||
_EN_PD_ANCHORS = (
|
||||
"In the Pratyantar Dasha of ",
|
||||
" in the Antar Dasha of ",
|
||||
"'s significations give the short-period result inside the ",
|
||||
" background.",
|
||||
)
|
||||
|
||||
|
||||
def _line_has_anchors(line: str, anchors: tuple[str, ...]) -> bool:
|
||||
return all(anchor in line for anchor in anchors)
|
||||
|
||||
|
||||
def _thicken_zh_dasha_density_fast(markdown: str) -> str:
|
||||
"""Same AD/PD inserts as the legacy scan, without searching inert lines."""
|
||||
lines = markdown.splitlines()
|
||||
out: list[str] = []
|
||||
for index, line in enumerate(lines):
|
||||
out.append(line)
|
||||
# Both patterns are case-sensitive and every hit contains these literals.
|
||||
ad_possible = _line_has_anchors(line, _ZH_AD_ANCHORS)
|
||||
pd_possible = _line_has_anchors(line, _ZH_PD_ANCHORS)
|
||||
if not ad_possible and not pd_possible:
|
||||
continue
|
||||
lookahead = "\n".join(lines[index + 1:index + 6])
|
||||
if "这一子运不是单独结论" in lookahead or "这个次子运只承担短周期细化" in lookahead:
|
||||
continue
|
||||
if ad_possible:
|
||||
ad_match = _ZH_AD_DENSITY_RE.search(line)
|
||||
if ad_match:
|
||||
md_lord, ad_lord, theme_lord = ad_match.groups()
|
||||
out.extend([
|
||||
"",
|
||||
(
|
||||
f"这一子运不是单独结论,而是把{theme_lord}的自然象征放进{md_lord}主大运的背景里筛选。"
|
||||
f"阅读时先看{ad_lord}本身的落宫、守护宫、尊贵状态与星宿,再看它和主大运星之间是否互相支持。"
|
||||
),
|
||||
"",
|
||||
(
|
||||
"若同一宫位或同一行星在分盘、Ashtakavarga、年度盘和行运中重复出现,"
|
||||
"该主题的可见度会提高;若这些层彼此冲突,则应把结果视为阶段性倾向,而不是孤立断语。"
|
||||
),
|
||||
])
|
||||
continue
|
||||
if pd_possible:
|
||||
pd_match = _ZH_PD_DENSITY_RE.search(line)
|
||||
if pd_match:
|
||||
ad_lord, pd_lord, md_lord, theme_lord = pd_match.groups()
|
||||
out.extend([
|
||||
"",
|
||||
(
|
||||
f"这个次子运只承担短周期细化:{pd_lord}会把{theme_lord}的具体征象带入{ad_lord}子运,"
|
||||
f"并受{md_lord}主背景限制。"
|
||||
),
|
||||
"",
|
||||
(
|
||||
"因此这里优先用于判断事情推进的节奏、触发点和轻重缓急;实际领域仍需回到本命落宫、"
|
||||
"分盘重复、年度盘和当前行运共同核对。"
|
||||
),
|
||||
])
|
||||
continue
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def _thicken_zh_dasha_density(markdown: str) -> str:
|
||||
"""Restore reader-facing density for translated AD/PD prose.
|
||||
|
||||
This adds reading instructions tied to already-present fields. It does not
|
||||
introduce new predictions or alter dates.
|
||||
"""
|
||||
if report_render_fast_enabled():
|
||||
return _thicken_zh_dasha_density_fast(markdown)
|
||||
lines = markdown.splitlines()
|
||||
out: list[str] = []
|
||||
for index, line in enumerate(lines):
|
||||
@@ -941,9 +1045,64 @@ def normalize_en_markdown_terms(markdown: str) -> str:
|
||||
return apply_en_residual_han(_thicken_en_dasha_density(text))
|
||||
|
||||
|
||||
def _thicken_en_dasha_density_fast(markdown: str) -> str:
|
||||
"""Same English AD inserts as the legacy scan, without searching inert lines."""
|
||||
lines = markdown.splitlines()
|
||||
out: list[str] = []
|
||||
current_md = None
|
||||
ad_context_added = False
|
||||
for index, line in enumerate(lines):
|
||||
md_heading = _EN_MD_HEADING_RE.match(line)
|
||||
if md_heading:
|
||||
next_md = md_heading.group(1)
|
||||
if next_md != current_md:
|
||||
current_md = next_md
|
||||
ad_context_added = False
|
||||
out.append(line)
|
||||
# Both patterns are case-sensitive and every hit contains these literals.
|
||||
ad_possible = _line_has_anchors(line, _EN_AD_ANCHORS)
|
||||
pd_possible = _line_has_anchors(line, _EN_PD_ANCHORS)
|
||||
if not ad_possible and not pd_possible:
|
||||
continue
|
||||
lookahead = "\n".join(lines[index + 1:index + 6])
|
||||
if (
|
||||
"This Antar Dasha is not read as a standalone verdict" in lookahead
|
||||
or "This Pratyantar Dasha is the short-cycle refinement" in lookahead
|
||||
):
|
||||
continue
|
||||
if ad_possible:
|
||||
ad_match = _EN_AD_DENSITY_RE.search(line)
|
||||
if ad_match and not ad_context_added:
|
||||
ad_lord, theme_lord, md_lord = ad_match.groups()
|
||||
out.extend([
|
||||
"",
|
||||
(
|
||||
f"This Antar Dasha is not read as a standalone verdict. It places {theme_lord}'s "
|
||||
f"natural significations inside the background of the {md_lord} Maha Dasha, then "
|
||||
f"filters them through {ad_lord}'s house placement, house ownership, dignity, "
|
||||
"nakshatra lord, conjunctions, and divisional-chart repetition."
|
||||
),
|
||||
"",
|
||||
(
|
||||
"When the same planet or house repeats through divisional charts, Ashtakavarga, "
|
||||
"annual factors, and transits, the theme becomes more visible. When those layers "
|
||||
"conflict, the result should be read as a staged tendency rather than an isolated statement."
|
||||
),
|
||||
])
|
||||
ad_context_added = True
|
||||
continue
|
||||
if pd_possible and _EN_PD_DENSITY_RE.search(line):
|
||||
# The row already contains the concrete PD lord and timing context.
|
||||
# Keep it, but do not append the same methodology paragraph per node.
|
||||
continue
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def _thicken_en_dasha_density(markdown: str) -> str:
|
||||
"""Add chapter-level AD context without repeating boilerplate on every PD row."""
|
||||
|
||||
if report_render_fast_enabled():
|
||||
return _thicken_en_dasha_density_fast(markdown)
|
||||
lines = markdown.splitlines()
|
||||
out: list[str] = []
|
||||
current_md = None
|
||||
|
||||
@@ -8,6 +8,9 @@ calculation, and delegates packet assembly/rendering to ``jyotish_engine``.
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime
|
||||
from importlib import import_module
|
||||
from types import SimpleNamespace
|
||||
@@ -32,6 +35,11 @@ except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution path
|
||||
from reader_dasha_applicability import render_dasha_applicability
|
||||
from report_density_packet import report_density_packet
|
||||
|
||||
try:
|
||||
from scripts.report_perf_flags import report_reference_cache_enabled
|
||||
except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution path
|
||||
from report_perf_flags import report_reference_cache_enabled
|
||||
|
||||
|
||||
class ProfessionalReportReferenceInputError(ValueError):
|
||||
"""The professional-reference request is outside the public contract."""
|
||||
@@ -187,18 +195,49 @@ def _export_args(birth: dict[str, Any]) -> SimpleNamespace:
|
||||
return SimpleNamespace(**payload)
|
||||
|
||||
|
||||
def build_professional_report_reference(handler, body: dict[str, Any], *, engine=None) -> dict[str, Any]:
|
||||
"""Build one JSON or Markdown response from one reused full-reading result."""
|
||||
if not isinstance(body, dict):
|
||||
raise ProfessionalReportReferenceInputError("request body must be an object")
|
||||
output_format = _normalize_format(body.get("format"))
|
||||
edition = _normalize_edition(body.get("edition"))
|
||||
packs = _normalize_packs(body.get("packs"))
|
||||
languages = _normalize_languages(body.get("languages"))
|
||||
if languages and (output_format != "markdown" or edition != FULL_DATA_EDITION):
|
||||
raise ProfessionalReportReferenceInputError("languages requires format markdown and edition full_data")
|
||||
def _load_report_cache():
|
||||
try:
|
||||
from scripts import consult_full_data_cache as cache
|
||||
except ModuleNotFoundError: # pragma: no cover - direct scripts/ execution path
|
||||
import consult_full_data_cache as cache
|
||||
return cache
|
||||
|
||||
birth = handler._high_rigor_birth_payload(body)
|
||||
|
||||
def _report_cache_sha() -> str:
|
||||
return os.environ.get("GITHUB_SHA") or ""
|
||||
|
||||
|
||||
def _report_cache_identity(
|
||||
birth: dict[str, Any],
|
||||
output_format: str,
|
||||
edition: str,
|
||||
packs: list[str],
|
||||
languages: list[str],
|
||||
include_fact_tables: bool,
|
||||
sha: str,
|
||||
) -> dict[str, Any]:
|
||||
return {
|
||||
"birth": birth,
|
||||
"format": output_format,
|
||||
"edition": edition,
|
||||
"packs": list(packs),
|
||||
"languages": list(languages),
|
||||
"include_fact_tables": include_fact_tables,
|
||||
"report_version": REPORT_VERSION_FULL,
|
||||
"github_sha": sha,
|
||||
}
|
||||
|
||||
|
||||
def _assemble_professional_report_reference(
|
||||
handler,
|
||||
birth: dict[str, Any],
|
||||
output_format: str,
|
||||
edition: str,
|
||||
packs: list[str],
|
||||
languages: list[str],
|
||||
include_fact_tables: bool,
|
||||
engine,
|
||||
) -> dict[str, Any]:
|
||||
full_reading = handler._compute_full_reading_for_thematic(birth)
|
||||
resolved_engine = engine or _load_engine()
|
||||
try:
|
||||
@@ -217,7 +256,7 @@ def build_professional_report_reference(handler, body: dict[str, Any], *, engine
|
||||
if edition == FULL_DATA_EDITION:
|
||||
# English renders from its own deep copy first, so nothing the Chinese
|
||||
# renderer does to the packet can reach it (and vice versa).
|
||||
english = _english_edition(packet, body.get("include_fact_tables") is True) if "en" in languages else {}
|
||||
english = _english_edition(packet, include_fact_tables) if "en" in languages else {}
|
||||
markdown = render_full_data_markdown(packet)
|
||||
else:
|
||||
markdown = resolved_engine.render_pl9_markdown(packet)
|
||||
@@ -228,7 +267,7 @@ def build_professional_report_reference(handler, body: dict[str, Any], *, engine
|
||||
"markdown": markdown,
|
||||
**({"fact_table_packet": report_density_packet(packet),
|
||||
"reader_dasha_applicability": clean_reader_appendix_markdown(render_dasha_applicability(packet))}
|
||||
if body.get("include_fact_tables") is True else {}),
|
||||
if include_fact_tables else {}),
|
||||
**(english if edition == FULL_DATA_EDITION and "en" in languages else {}),
|
||||
}
|
||||
return {
|
||||
@@ -236,3 +275,49 @@ def build_professional_report_reference(handler, body: dict[str, Any], *, engine
|
||||
"edition": edition,
|
||||
"report": packet,
|
||||
}
|
||||
|
||||
|
||||
def build_professional_report_reference(handler, body: dict[str, Any], *, engine=None) -> dict[str, Any]:
|
||||
"""Build one JSON or Markdown response from one reused full-reading result."""
|
||||
if not isinstance(body, dict):
|
||||
raise ProfessionalReportReferenceInputError("request body must be an object")
|
||||
output_format = _normalize_format(body.get("format"))
|
||||
edition = _normalize_edition(body.get("edition"))
|
||||
packs = _normalize_packs(body.get("packs"))
|
||||
languages = _normalize_languages(body.get("languages"))
|
||||
if languages and (output_format != "markdown" or edition != FULL_DATA_EDITION):
|
||||
raise ProfessionalReportReferenceInputError("languages requires format markdown and edition full_data")
|
||||
|
||||
birth = handler._high_rigor_birth_payload(body)
|
||||
include_fact_tables = body.get("include_fact_tables") is True
|
||||
sha = _report_cache_sha()
|
||||
use_cache = (
|
||||
report_reference_cache_enabled()
|
||||
and output_format == "markdown"
|
||||
and edition == FULL_DATA_EDITION
|
||||
and sha not in ("", "unknown")
|
||||
)
|
||||
if not use_cache:
|
||||
return _assemble_professional_report_reference(
|
||||
handler, birth, output_format, edition, packs, languages, include_fact_tables, engine,
|
||||
)
|
||||
identity = _report_cache_identity(
|
||||
birth, output_format, edition, packs, languages, include_fact_tables, sha,
|
||||
)
|
||||
started = time.perf_counter()
|
||||
|
||||
def builder(_identity: dict[str, Any]) -> dict[str, Any]:
|
||||
return _assemble_professional_report_reference(
|
||||
handler, birth, output_format, edition, packs, languages, include_fact_tables, engine,
|
||||
)
|
||||
|
||||
cached = _load_report_cache().get_or_build(
|
||||
identity,
|
||||
builder,
|
||||
cache_name="report_reference_cache",
|
||||
json_default=str,
|
||||
)
|
||||
elapsed_ms = int(round((time.perf_counter() - started) * 1000))
|
||||
state = "hit" if cached["hit"] else "miss"
|
||||
print(f"[report-reference-cache] {state} elapsed_ms={elapsed_ms}", file=sys.stderr)
|
||||
return cached["packet"]
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Call-time switches for long-report rendering and same-chart reuse.
|
||||
|
||||
Values are read on every call so one process can measure the old path and the
|
||||
new path back to back. The old path stays in place when a switch is off.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
# Masked Windows A/B on 2026-10-14. Render shortcuts are byte-identical and use
|
||||
# less memory, but the end-to-end drop was 20–23%, short of the 35% gate, so
|
||||
# the fast path stays off. Cache hits matched the miss bytes and returned in
|
||||
# about 0.06s, under the 1.5s gate. A missing or unknown GITHUB_SHA still
|
||||
# disables the cache at the call site.
|
||||
RENDER_FAST_DEFAULT = False
|
||||
REFERENCE_CACHE_DEFAULT = True
|
||||
|
||||
_ON = {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def _enabled(name: str, default: bool) -> bool:
|
||||
raw = os.environ.get(name)
|
||||
if raw is None or not raw.strip():
|
||||
return default
|
||||
return raw.strip().lower() in _ON
|
||||
|
||||
|
||||
def report_render_fast_enabled() -> bool:
|
||||
"""True when the equivalent render and hash shortcuts may run."""
|
||||
return _enabled("JYOTISH_REPORT_RENDER_FAST", RENDER_FAST_DEFAULT)
|
||||
|
||||
|
||||
def report_reference_cache_enabled() -> bool:
|
||||
"""True when a full_data markdown response may be reused from disk.
|
||||
|
||||
A missing or ``unknown`` ``GITHUB_SHA`` still disables the cache at the
|
||||
call site. This function only reads the switch.
|
||||
"""
|
||||
return _enabled("JYOTISH_REPORT_REFERENCE_CACHE", REFERENCE_CACHE_DEFAULT)
|
||||
@@ -0,0 +1,428 @@
|
||||
"""A/B timing for the professional long report.
|
||||
|
||||
A is both switches off. B is both on. Same process, fictional charts only.
|
||||
A field sha mismatch exits 1 and prints a short diff context, not the report.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import tracemalloc
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
for _folder in (ROOT / "scripts", ROOT / "jyotish_vedic", ROOT):
|
||||
_text = str(_folder)
|
||||
if _text not in sys.path:
|
||||
sys.path.insert(0, _text)
|
||||
|
||||
PRODUCTION_BODY = {
|
||||
"format": "markdown",
|
||||
"edition": "full_data",
|
||||
"include_fact_tables": True,
|
||||
"languages": ["zh", "en"],
|
||||
}
|
||||
SECTION2_CHART = {
|
||||
"year": 1990,
|
||||
"month": 1,
|
||||
"day": 1,
|
||||
"hour": 12,
|
||||
"minute": 0,
|
||||
"second": 0,
|
||||
"lat": 28.6139,
|
||||
"lon": 77.2090,
|
||||
"tz": 5.5,
|
||||
"node_mode": "mean",
|
||||
"ayanamsa": "lahiri",
|
||||
"today": "2026-10-13",
|
||||
"transit_date": "2026-10-13",
|
||||
"target_year": 2026,
|
||||
"age": 36,
|
||||
"birth_time_accuracy": "confirmed",
|
||||
}
|
||||
FIELDS = (
|
||||
"response",
|
||||
"markdown",
|
||||
"markdown_en",
|
||||
"fact_table_packet",
|
||||
"reader_dasha_applicability",
|
||||
)
|
||||
|
||||
|
||||
def _mask_pyjhora() -> str:
|
||||
root = Path(tempfile.mkdtemp(prefix="nojhora-"))
|
||||
package = root / "jhora"
|
||||
package.mkdir()
|
||||
(package / "__init__.py").write_text('raise ImportError("masked")\n', encoding="utf-8")
|
||||
sys.path.insert(0, str(root))
|
||||
prior = os.environ.get("PYTHONPATH", "")
|
||||
os.environ["PYTHONPATH"] = str(root) + (os.pathsep + prior if prior else "")
|
||||
for name in list(sys.modules):
|
||||
if name == "jhora" or name.startswith("jhora."):
|
||||
del sys.modules[name]
|
||||
return str(root)
|
||||
|
||||
|
||||
def _sha(text: str) -> str:
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _canonical(value) -> str:
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
return json.dumps(value, ensure_ascii=False, default=str)
|
||||
|
||||
|
||||
def _field_texts(result: dict) -> dict[str, str]:
|
||||
whole = json.dumps(result, ensure_ascii=False, default=str)
|
||||
return {
|
||||
"response": whole,
|
||||
"markdown": _canonical(result.get("markdown", "")),
|
||||
"markdown_en": _canonical(result.get("markdown_en", "")),
|
||||
"fact_table_packet": _canonical(result.get("fact_table_packet")),
|
||||
"reader_dasha_applicability": _canonical(result.get("reader_dasha_applicability", "")),
|
||||
}
|
||||
|
||||
|
||||
def _field_shas(result: dict) -> dict[str, str]:
|
||||
return {name: _sha(text) for name, text in _field_texts(result).items()}
|
||||
|
||||
|
||||
def _diff_context(left: str, right: str) -> str:
|
||||
limit = min(len(left), len(right))
|
||||
index = next((i for i in range(limit) if left[i] != right[i]), limit)
|
||||
start = max(0, index - 40)
|
||||
stop = index + 40
|
||||
return (
|
||||
f"at {index} len {len(left)} vs {len(right)} "
|
||||
f"A={left[start:stop]!r} B={right[start:stop]!r}"
|
||||
)
|
||||
|
||||
|
||||
def _median(values: list[float]) -> float:
|
||||
ordered = sorted(values)
|
||||
return ordered[len(ordered) // 2]
|
||||
|
||||
|
||||
class _Bucket:
|
||||
def __init__(self) -> None:
|
||||
self.cmd = 0.0
|
||||
self.packet = 0.0
|
||||
self.zh = 0.0
|
||||
self.en = 0.0
|
||||
self.hash = 0.0
|
||||
|
||||
def reset(self) -> None:
|
||||
self.cmd = self.packet = self.zh = self.en = self.hash = 0.0
|
||||
|
||||
def as_dict(self) -> dict[str, float]:
|
||||
return {
|
||||
"cmd_full_reading_s": round(self.cmd, 4),
|
||||
"packet_s": round(self.packet, 4),
|
||||
"zh_render_s": round(self.zh, 4),
|
||||
"en_render_s": round(self.en, 4),
|
||||
"hash_s": round(self.hash, 4),
|
||||
}
|
||||
|
||||
|
||||
def _install_timers(bucket: _Bucket) -> None:
|
||||
import scripts.calculation_profile_contract as contract
|
||||
import scripts.full_report_quality_gate as gate
|
||||
import scripts.jyotish_engine as engine
|
||||
import scripts.professional_report_reference as reference
|
||||
|
||||
orig_packet = engine.build_professional_report_reference_packet
|
||||
|
||||
def timed_packet(*args, **kwargs):
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
return orig_packet(*args, **kwargs)
|
||||
finally:
|
||||
bucket.packet += time.perf_counter() - started
|
||||
|
||||
engine.build_professional_report_reference_packet = timed_packet
|
||||
orig_render = reference.render_full_data_markdown
|
||||
|
||||
def timed_render(packet, *args, **kwargs):
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
return orig_render(packet, *args, **kwargs)
|
||||
finally:
|
||||
elapsed = time.perf_counter() - started
|
||||
if isinstance(packet, dict) and packet.get("report_language") == "en":
|
||||
bucket.en += elapsed
|
||||
else:
|
||||
bucket.zh += elapsed
|
||||
|
||||
reference.render_full_data_markdown = timed_render
|
||||
orig_hash = contract.hash_bound_result
|
||||
|
||||
def timed_hash(*args, **kwargs):
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
return orig_hash(*args, **kwargs)
|
||||
finally:
|
||||
bucket.hash += time.perf_counter() - started
|
||||
|
||||
contract.hash_bound_result = timed_hash
|
||||
gate.hash_bound_result = timed_hash
|
||||
|
||||
|
||||
def _set_flags(fast: bool, cache: bool) -> None:
|
||||
os.environ["JYOTISH_REPORT_RENDER_FAST"] = "1" if fast else "0"
|
||||
os.environ["JYOTISH_REPORT_REFERENCE_CACHE"] = "1" if cache else "0"
|
||||
|
||||
|
||||
class _Handler:
|
||||
def __init__(self, birth: dict, bucket: _Bucket) -> None:
|
||||
self.birth = birth
|
||||
self.bucket = bucket
|
||||
|
||||
def _high_rigor_birth_payload(self, _body: dict) -> dict:
|
||||
return dict(self.birth)
|
||||
|
||||
def _compute_full_reading_for_thematic(self, birth: dict):
|
||||
from scripts.jyotish_engine import cmd_full_reading
|
||||
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
return cmd_full_reading(SimpleNamespace(**birth))
|
||||
finally:
|
||||
self.bucket.cmd += time.perf_counter() - started
|
||||
|
||||
|
||||
def _run_once(birth: dict, body: dict, bucket: _Bucket, handler=None) -> dict:
|
||||
from scripts.professional_report_reference import build_professional_report_reference
|
||||
|
||||
bucket.reset()
|
||||
tracemalloc.start()
|
||||
started = time.perf_counter()
|
||||
result = build_professional_report_reference(handler or _Handler(birth, bucket), body)
|
||||
elapsed = time.perf_counter() - started
|
||||
_current, peak = tracemalloc.get_traced_memory()
|
||||
tracemalloc.stop()
|
||||
texts = _field_texts(result)
|
||||
return {
|
||||
"elapsed_s": round(elapsed, 4),
|
||||
"peak_bytes": peak,
|
||||
"response_bytes": len(texts["response"].encode("utf-8")),
|
||||
"segments": bucket.as_dict(),
|
||||
"sha": _field_shas(result),
|
||||
"_texts": texts,
|
||||
}
|
||||
|
||||
|
||||
def _repeat(birth: dict, body: dict, bucket: _Bucket, times: int) -> list[dict]:
|
||||
return [_run_once(birth, body, bucket) for _ in range(times)]
|
||||
|
||||
|
||||
def _public_row(row: dict) -> dict:
|
||||
return {key: value for key, value in row.items() if key != "_texts"}
|
||||
|
||||
|
||||
def _compare(label: str, left: dict, right: dict, problems: list[str]) -> None:
|
||||
left_texts = left["_texts"]
|
||||
right_texts = right["_texts"]
|
||||
for name in FIELDS:
|
||||
if left["sha"][name] != right["sha"][name]:
|
||||
problems.append(f"{label} {name} {_diff_context(left_texts[name], right_texts[name])}")
|
||||
|
||||
|
||||
def _freeze_clocks():
|
||||
"""Pin datetime.now / utcnow in every already imported module.
|
||||
|
||||
``format=json`` and the reference edition embed wall-clock fields. Freezing
|
||||
only the engine module leaves other imports on the real clock.
|
||||
"""
|
||||
import datetime as dt_mod
|
||||
|
||||
real = dt_mod.datetime
|
||||
fixed_utc = real(2026, 10, 13, 0, 0, 0)
|
||||
fixed_local = real(2026, 10, 13, 8, 0, 0)
|
||||
|
||||
class Frozen(real):
|
||||
@classmethod
|
||||
def now(cls, tz=None):
|
||||
if tz is not None:
|
||||
return fixed_utc.replace(tzinfo=tz)
|
||||
return fixed_local
|
||||
|
||||
@classmethod
|
||||
def utcnow(cls):
|
||||
return fixed_utc
|
||||
|
||||
saved = []
|
||||
for module in list(sys.modules.values()):
|
||||
if module is None:
|
||||
continue
|
||||
current = getattr(module, "datetime", None)
|
||||
if isinstance(current, type) and current.__name__ == "datetime":
|
||||
saved.append((module, current))
|
||||
module.datetime = Frozen
|
||||
return saved
|
||||
|
||||
|
||||
def _restore_clocks(saved) -> None:
|
||||
for module, current in saved:
|
||||
module.datetime = current
|
||||
|
||||
|
||||
def _strip_generated_at(value):
|
||||
if isinstance(value, dict):
|
||||
return {key: _strip_generated_at(item) for key, item in value.items() if key != "generated_at"}
|
||||
if isinstance(value, list):
|
||||
return [_strip_generated_at(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def _sha_pair(birth: dict, body: dict, bucket: _Bucket) -> dict:
|
||||
"""Compare one frozen clock on one reused full reading.
|
||||
|
||||
A second ``cmd_full_reading`` appends onto shared trait lists, so two live
|
||||
engine calls are not a byte test of the render switch.
|
||||
"""
|
||||
import copy
|
||||
|
||||
from scripts.jyotish_engine import cmd_full_reading
|
||||
|
||||
_set_flags(False, False)
|
||||
reading = cmd_full_reading(SimpleNamespace(**birth))
|
||||
|
||||
class _Reuse:
|
||||
def _high_rigor_birth_payload(self, _body: dict) -> dict:
|
||||
return dict(birth)
|
||||
|
||||
def _compute_full_reading_for_thematic(self, _birth: dict):
|
||||
return copy.deepcopy(reading)
|
||||
|
||||
_set_flags(False, False)
|
||||
left = _run_once(birth, body, bucket, _Reuse())
|
||||
_set_flags(True, False)
|
||||
right = _run_once(birth, body, bucket, _Reuse())
|
||||
problems = []
|
||||
_compare("side", left, right, problems)
|
||||
compared = "full"
|
||||
if problems:
|
||||
left_stripped = _sha(_canonical(_strip_generated_at(json.loads(left["_texts"]["response"]))))
|
||||
right_stripped = _sha(_canonical(_strip_generated_at(json.loads(right["_texts"]["response"]))))
|
||||
if left_stripped == right_stripped:
|
||||
problems.clear()
|
||||
compared = "without_generated_at"
|
||||
else:
|
||||
compared = "mismatch"
|
||||
return {
|
||||
"compared": compared,
|
||||
"sha_a": left["sha"]["response"],
|
||||
"sha_b": right["sha"]["response"],
|
||||
"problems": problems,
|
||||
}
|
||||
|
||||
|
||||
def _charts() -> dict[str, dict]:
|
||||
from tests.test_report_english_edition import CASES
|
||||
|
||||
return {
|
||||
"day": {key: value for key, value in CASES["day"].items() if key != "name"},
|
||||
"night": {key: value for key, value in CASES["night"].items() if key != "name"},
|
||||
"south": {key: value for key, value in CASES["south"].items() if key != "name"},
|
||||
"section2": dict(SECTION2_CHART),
|
||||
}
|
||||
|
||||
|
||||
def _clear_report_cache() -> None:
|
||||
folder = ROOT / "scratch" / "local" / "report_reference_cache"
|
||||
if not folder.is_dir():
|
||||
return
|
||||
for path in folder.glob("*.json"):
|
||||
path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--mask-pyjhora", action="store_true")
|
||||
parser.add_argument("--only-a", action="store_true")
|
||||
parser.add_argument("--side-only", action="store_true")
|
||||
args = parser.parse_args()
|
||||
mask_path = _mask_pyjhora() if args.mask_pyjhora else ""
|
||||
bucket = _Bucket()
|
||||
_install_timers(bucket)
|
||||
os.environ.pop("GITHUB_SHA", None)
|
||||
report: dict = {
|
||||
"python": platform.python_version(),
|
||||
"platform": platform.platform(),
|
||||
"mask_pyjhora": bool(args.mask_pyjhora),
|
||||
"mask_path": mask_path,
|
||||
"only_a": args.only_a,
|
||||
"charts": {},
|
||||
}
|
||||
problems: list[str] = []
|
||||
chart_items = () if args.side_only else _charts().items()
|
||||
for label, birth in chart_items:
|
||||
_set_flags(False, False)
|
||||
arm_a = _repeat(birth, PRODUCTION_BODY, bucket, 3)
|
||||
entry: dict = {"A": [_public_row(row) for row in arm_a]}
|
||||
if not args.only_a:
|
||||
_set_flags(True, False)
|
||||
arm_b = _repeat(birth, PRODUCTION_BODY, bucket, 3)
|
||||
for row in arm_b:
|
||||
_compare(f"{label} B-miss", arm_a[0], row, problems)
|
||||
os.environ["GITHUB_SHA"] = "report-reference-perf-ab"
|
||||
_set_flags(True, True)
|
||||
_clear_report_cache()
|
||||
miss = _run_once(birth, PRODUCTION_BODY, bucket)
|
||||
_compare(f"{label} cache-miss", arm_a[0], miss, problems)
|
||||
hits = _repeat(birth, PRODUCTION_BODY, bucket, 3)
|
||||
for row in hits:
|
||||
_compare(f"{label} cache-hit", miss, row, problems)
|
||||
entry["B_miss"] = [_public_row(row) for row in arm_b]
|
||||
entry["cache_miss"] = _public_row(miss)
|
||||
entry["B_hit"] = [_public_row(row) for row in hits]
|
||||
os.environ.pop("GITHUB_SHA", None)
|
||||
report["charts"][label] = entry
|
||||
print(f"{label} done", file=sys.stderr)
|
||||
if not args.only_a:
|
||||
saved_clocks = _freeze_clocks()
|
||||
try:
|
||||
sample = _charts()["section2"]
|
||||
report["format_json"] = _sha_pair(sample, {"format": "json", "edition": "full_data"}, bucket)
|
||||
report["edition_reference"] = _sha_pair(
|
||||
sample, {"format": "markdown", "edition": "reference"}, bucket,
|
||||
)
|
||||
finally:
|
||||
_restore_clocks(saved_clocks)
|
||||
problems.extend(report["format_json"]["problems"])
|
||||
problems.extend(report["edition_reference"]["problems"])
|
||||
_set_flags(False, False)
|
||||
from scripts.calculation_profile_contract import hash_bound_result
|
||||
from scripts.jyotish_engine import build_professional_report_reference_packet, cmd_full_reading
|
||||
from scripts.professional_report_reference import _export_args
|
||||
|
||||
reading = cmd_full_reading(SimpleNamespace(**sample))
|
||||
packet = build_professional_report_reference_packet(reading, _export_args(sample), [])
|
||||
input_hash = packet["calculation_profile"]["input_hash"]
|
||||
|
||||
def _time_hash(fast: bool) -> float:
|
||||
_set_flags(fast, False)
|
||||
started = time.perf_counter()
|
||||
hash_bound_result(packet, input_hash)
|
||||
return time.perf_counter() - started
|
||||
|
||||
off = [_time_hash(False) for _ in range(3)]
|
||||
on = [_time_hash(True) for _ in range(3)]
|
||||
report["hash_once_s"] = {"off": [round(item, 4) for item in off], "on": [round(item, 4) for item in on]}
|
||||
report["problems"] = problems
|
||||
print(json.dumps(report, ensure_ascii=False))
|
||||
return 1 if problems else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,374 @@
|
||||
"""Byte-identical report shortcuts and the full_data markdown cache."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from scripts.calculation_profile_contract import hash_bound_result
|
||||
from scripts.pl9_full_data_export import render_full_data_markdown
|
||||
from scripts.pl9_language_terms import _thicken_zh_dasha_density, replace_natural_language_term
|
||||
from scripts.report_perf_flags import (
|
||||
REFERENCE_CACHE_DEFAULT,
|
||||
RENDER_FAST_DEFAULT,
|
||||
report_reference_cache_enabled,
|
||||
report_render_fast_enabled,
|
||||
)
|
||||
from scripts.professional_report_reference import build_professional_report_reference
|
||||
from tests.test_report_english_edition import CASES, _packet
|
||||
|
||||
|
||||
def _legacy_drop_key_recursive(payload: Any, key_name: str) -> Any:
|
||||
if isinstance(payload, dict):
|
||||
return {
|
||||
key: _legacy_drop_key_recursive(value, key_name)
|
||||
for key, value in payload.items()
|
||||
if key != key_name
|
||||
}
|
||||
if isinstance(payload, list):
|
||||
return [_legacy_drop_key_recursive(item, key_name) for item in payload]
|
||||
return payload
|
||||
|
||||
|
||||
def _legacy_drop_dotted_path(payload: Any, path: tuple[str, ...]) -> Any:
|
||||
if len(path) == 2 and path[0] == "**":
|
||||
return _legacy_drop_key_recursive(payload, path[1])
|
||||
if not path or not isinstance(payload, dict) or path[0] not in payload:
|
||||
return payload
|
||||
cloned = dict(payload)
|
||||
key, *rest = path
|
||||
if not rest:
|
||||
cloned.pop(key, None)
|
||||
return cloned
|
||||
cloned[key] = _legacy_drop_dotted_path(payload[key], tuple(rest))
|
||||
return cloned
|
||||
|
||||
|
||||
def _legacy_canonicalize(value: Any) -> Any:
|
||||
if isinstance(value, dict):
|
||||
normalized: dict[str, Any] = {}
|
||||
for key, child in value.items():
|
||||
normalized_key = key if isinstance(key, str) else f"__{type(key).__name__}__:{key}"
|
||||
normalized[normalized_key] = _legacy_canonicalize(child)
|
||||
return normalized
|
||||
if isinstance(value, (list, tuple)):
|
||||
return [_legacy_canonicalize(item) for item in value]
|
||||
return value
|
||||
|
||||
|
||||
def _legacy_payload(result: dict, scope: dict) -> dict:
|
||||
excluded_top = {"result_hash", "result_binding", "calculation_profile", "calculation_profile_id"} | {
|
||||
str(key) for key in (scope.get("excluded_top_keys") or ())
|
||||
}
|
||||
payload = {key: value for key, value in result.items() if key not in excluded_top}
|
||||
for raw_path in scope.get("excluded_paths") or ():
|
||||
parts = tuple(part for part in str(raw_path).split(".") if part)
|
||||
if parts:
|
||||
payload = _legacy_drop_dotted_path(payload, parts)
|
||||
return payload
|
||||
|
||||
|
||||
def _legacy_hash(result: dict, input_hash: str, scope: dict) -> str:
|
||||
encoded = json.dumps(
|
||||
_legacy_canonicalize({"input_hash": input_hash, "result": _legacy_payload(result, scope)}),
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
)
|
||||
return hashlib.sha256(encoded.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def day_views():
|
||||
"""One real day packet. ``delivered`` is the bound object; render uses a copy."""
|
||||
from scripts.jyotish_engine import build_professional_report_reference_packet, cmd_full_reading
|
||||
from scripts.professional_report_reference import _export_args
|
||||
|
||||
os.environ["JYOTISH_REPORT_RENDER_FAST"] = "0"
|
||||
os.environ["JYOTISH_REPORT_REFERENCE_CACHE"] = "0"
|
||||
try:
|
||||
case = CASES["day"]
|
||||
reading = cmd_full_reading(type("Args", (), dict(case))())
|
||||
delivered = build_professional_report_reference_packet(reading, _export_args(dict(case)), [])
|
||||
packet = dict(delivered)
|
||||
packet["report_version"] = "pl9_personal_long_report.v3"
|
||||
zh = render_full_data_markdown(copy.deepcopy(packet))
|
||||
english = copy.deepcopy(packet)
|
||||
english["report_language"] = "en"
|
||||
en = render_full_data_markdown(english)
|
||||
finally:
|
||||
os.environ.pop("JYOTISH_REPORT_RENDER_FAST", None)
|
||||
os.environ.pop("JYOTISH_REPORT_REFERENCE_CACHE", None)
|
||||
return delivered, packet, zh, en
|
||||
|
||||
|
||||
def test_flags_follow_the_environment_at_call_time(monkeypatch) -> None:
|
||||
monkeypatch.delenv("JYOTISH_REPORT_RENDER_FAST", raising=False)
|
||||
monkeypatch.delenv("JYOTISH_REPORT_REFERENCE_CACHE", raising=False)
|
||||
assert report_render_fast_enabled() is RENDER_FAST_DEFAULT
|
||||
assert report_reference_cache_enabled() is REFERENCE_CACHE_DEFAULT
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0")
|
||||
monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "0")
|
||||
assert report_render_fast_enabled() is False
|
||||
assert report_reference_cache_enabled() is False
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1")
|
||||
monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "1")
|
||||
assert report_render_fast_enabled() is True
|
||||
assert report_reference_cache_enabled() is True
|
||||
|
||||
|
||||
def test_fast_render_matches_the_legacy_bytes(day_views, monkeypatch) -> None:
|
||||
_delivered, packet, zh_off, en_off = day_views
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1")
|
||||
monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "0")
|
||||
zh_on = render_full_data_markdown(copy.deepcopy(packet))
|
||||
english = copy.deepcopy(packet)
|
||||
english["report_language"] = "en"
|
||||
en_on = render_full_data_markdown(english)
|
||||
assert zh_on == zh_off
|
||||
assert en_on == en_off
|
||||
|
||||
|
||||
def _both(function, monkeypatch, *args):
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0")
|
||||
off = function(*args)
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1")
|
||||
on = function(*args)
|
||||
assert on == off
|
||||
return off
|
||||
|
||||
|
||||
def test_dasha_density_samples_come_from_the_real_render(day_views, monkeypatch) -> None:
|
||||
_delivered, _packet_view, zh, _en = day_views
|
||||
lines = [line for line in zh.splitlines() if line.strip()]
|
||||
# This fictional chart never emits the density sentence. Fill the existing
|
||||
# sanitizer sentence with lords that do appear in the render.
|
||||
lords = [token for token in ("木星", "北交点", "土星", "金星") if token in zh]
|
||||
assert len(lords) >= 2
|
||||
hit = f"{lords[0]}主大运中的{lords[1]}子运,会把{lords[0]}的自然主题带入当前阶段。"
|
||||
miss = next(line for line in lines if "子运" not in line and "`" not in line)
|
||||
quoted = next(line for line in lines if "`" in line)
|
||||
hit_out = _both(_thicken_zh_dasha_density, monkeypatch, hit + "\n")
|
||||
assert "这一子运不是单独结论" in hit_out or "这个次子运只承担短周期细化" in hit_out
|
||||
miss_out = _both(_thicken_zh_dasha_density, monkeypatch, miss + "\n")
|
||||
assert miss_out == miss
|
||||
quoted_out = _both(_thicken_zh_dasha_density, monkeypatch, quoted + "\n")
|
||||
assert "`" in quoted_out
|
||||
assert quoted in quoted_out
|
||||
|
||||
|
||||
def test_term_samples_come_from_the_real_render(day_views, monkeypatch) -> None:
|
||||
_delivered, _packet_view, zh, _en = day_views
|
||||
lines = zh.splitlines()
|
||||
raw = "Ashtakavarga"
|
||||
hit = next(line for line in lines if raw in re.sub(r"`[^`]*`", "", line))
|
||||
miss = next(line for line in lines if line.strip() and raw not in line)
|
||||
span = next(span for span in re.findall(r"`[^`]*`", zh) if raw not in span)
|
||||
hit_out = _both(replace_natural_language_term, monkeypatch, hit, raw, "探针词")
|
||||
assert "探针词" in hit_out or hit_out == hit
|
||||
if re.search(rf"(?<![A-Za-z0-9_]){raw}(?![A-Za-z0-9_])", re.sub(r"`[^`]*`", "", hit)):
|
||||
assert "探针词" in hit_out
|
||||
miss_out = _both(replace_natural_language_term, monkeypatch, miss, raw, "探针词")
|
||||
assert miss_out == miss
|
||||
quoted_out = _both(replace_natural_language_term, monkeypatch, span, raw, "探针词")
|
||||
assert quoted_out == span
|
||||
|
||||
|
||||
def test_fast_hash_matches_the_legacy_algorithm_on_a_real_packet(day_views, monkeypatch) -> None:
|
||||
packet, _render_packet, _zh, _en = day_views
|
||||
scope = packet["result_binding"]["binding_scope"]
|
||||
input_hash = packet["calculation_profile"]["input_hash"]
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "1")
|
||||
assert hash_bound_result(packet, input_hash, scope) == _legacy_hash(packet, input_hash, scope)
|
||||
assert hash_bound_result(packet, input_hash, scope) == packet["result_hash"]
|
||||
variant = copy.deepcopy(packet)
|
||||
variant["probe_layer"] = {
|
||||
"elapsed_seconds": 1.5,
|
||||
"called_at": "2001-01-01T00:00:00Z",
|
||||
"cache_created_at": "created",
|
||||
"cache_expires_at": "expires",
|
||||
"nested": {"elapsed_seconds": 9, (1, 2): {"called_at": "inside"}},
|
||||
}
|
||||
summary = variant.setdefault("ai_and_audit", {}).setdefault("summary", {})
|
||||
summary["elapsed_seconds"] = 99
|
||||
summary["kept"] = "yes"
|
||||
assert hash_bound_result(variant, input_hash, scope) == _legacy_hash(variant, input_hash, scope)
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0")
|
||||
assert hash_bound_result(packet, input_hash) == packet["result_hash"]
|
||||
|
||||
|
||||
class _CountingHandler:
|
||||
def __init__(self, birth: dict, packet: dict) -> None:
|
||||
self.birth = birth
|
||||
self.packet = packet
|
||||
self.calls = 0
|
||||
self.fail_once = False
|
||||
self.delay = 0.0
|
||||
self.lock = threading.Lock()
|
||||
|
||||
def _high_rigor_birth_payload(self, body: dict) -> dict:
|
||||
payload = dict(self.birth)
|
||||
if body.get("today"):
|
||||
payload["today"] = body["today"]
|
||||
payload["candidate_range"] = body.get("candidate_range")
|
||||
return payload
|
||||
|
||||
def _compute_full_reading_for_thematic(self, _birth: dict) -> dict:
|
||||
with self.lock:
|
||||
self.calls += 1
|
||||
call = self.calls
|
||||
if self.delay:
|
||||
time.sleep(self.delay)
|
||||
if self.fail_once and call == 1:
|
||||
raise RuntimeError("report_build_failed")
|
||||
return {"stub": True}
|
||||
|
||||
|
||||
def _body(**extra) -> dict:
|
||||
body = {
|
||||
"format": "markdown",
|
||||
"edition": "full_data",
|
||||
"include_fact_tables": True,
|
||||
"languages": ["zh", "en"],
|
||||
"today": "2026-04-07",
|
||||
}
|
||||
body.update(extra)
|
||||
return body
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def cached_report(monkeypatch, tmp_path, day_views):
|
||||
import scripts.consult_full_data_cache as cache
|
||||
import scripts.professional_report_reference as reference
|
||||
|
||||
def _dir(name: str = "consult_full_data_cache"):
|
||||
path = tmp_path / name
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
monkeypatch.setattr(cache, "cache_dir", _dir)
|
||||
monkeypatch.setattr(reference, "render_full_data_markdown", lambda packet: "en-body" if packet.get("report_language") == "en" else "zh-body")
|
||||
monkeypatch.setattr(reference, "report_density_packet", lambda _packet: {"facts": 1})
|
||||
monkeypatch.setattr(reference, "render_dasha_applicability", lambda _packet, language="zh": f"app-{language}")
|
||||
monkeypatch.setattr(reference, "clean_reader_appendix_markdown", lambda text: text)
|
||||
packet = day_views[1]
|
||||
birth = {key: value for key, value in CASES["day"].items() if key != "name"}
|
||||
handler = _CountingHandler(birth, packet)
|
||||
|
||||
class _Engine:
|
||||
def build_professional_report_reference_packet(self, _reading, _args, _packs):
|
||||
return copy.deepcopy(packet)
|
||||
|
||||
def render_pl9_markdown(self, _packet):
|
||||
return "reference"
|
||||
|
||||
monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "1")
|
||||
monkeypatch.setenv("JYOTISH_REPORT_RENDER_FAST", "0")
|
||||
monkeypatch.setenv("GITHUB_SHA", "report-cache-test")
|
||||
return handler, _Engine(), _dir("report_reference_cache")
|
||||
|
||||
|
||||
def _dumps(result: dict) -> str:
|
||||
return json.dumps(result, ensure_ascii=False, default=str)
|
||||
|
||||
|
||||
def test_second_call_reuses_the_cached_response(cached_report) -> None:
|
||||
handler, engine, folder = cached_report
|
||||
first = build_professional_report_reference(handler, _body(), engine=engine)
|
||||
second = build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert handler.calls == 1
|
||||
assert _dumps(first) == _dumps(second)
|
||||
assert list(folder.glob("*.json"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("extra", [
|
||||
{"today": "2026-05-01"},
|
||||
{"languages": ["zh"]},
|
||||
{"include_fact_tables": False},
|
||||
{"candidate_range": {"kind": "probe"}},
|
||||
])
|
||||
def test_changed_inputs_miss_the_report_cache(cached_report, extra) -> None:
|
||||
handler, engine, _folder = cached_report
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
build_professional_report_reference(handler, _body(**extra), engine=engine)
|
||||
assert handler.calls == 2
|
||||
|
||||
|
||||
def test_github_sha_is_part_of_the_report_cache_key(cached_report, monkeypatch) -> None:
|
||||
handler, engine, _folder = cached_report
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
monkeypatch.setenv("GITHUB_SHA", "report-cache-other")
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert handler.calls == 2
|
||||
|
||||
|
||||
def test_json_format_and_disabled_cache_write_nothing(cached_report, monkeypatch) -> None:
|
||||
handler, engine, folder = cached_report
|
||||
for path in folder.glob("*.json"):
|
||||
path.unlink()
|
||||
build_professional_report_reference(
|
||||
handler,
|
||||
{"format": "json", "edition": "full_data", "today": "2026-04-07"},
|
||||
engine=engine,
|
||||
)
|
||||
assert list(folder.glob("*.json")) == []
|
||||
monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "0")
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert list(folder.glob("*.json")) == []
|
||||
monkeypatch.setenv("JYOTISH_REPORT_REFERENCE_CACHE", "1")
|
||||
monkeypatch.setenv("GITHUB_SHA", "unknown")
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert list(folder.glob("*.json")) == []
|
||||
|
||||
|
||||
def test_concurrent_same_key_builds_once(cached_report) -> None:
|
||||
handler, engine, _folder = cached_report
|
||||
handler.delay = 0.2
|
||||
barrier = threading.Barrier(2)
|
||||
errors = []
|
||||
|
||||
def run() -> None:
|
||||
try:
|
||||
barrier.wait()
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
except Exception as exc: # pragma: no cover - recorded for the assertion
|
||||
errors.append(exc)
|
||||
|
||||
threads = [threading.Thread(target=run) for _ in range(2)]
|
||||
for thread in threads:
|
||||
thread.start()
|
||||
for thread in threads:
|
||||
thread.join()
|
||||
assert errors == []
|
||||
assert handler.calls == 1
|
||||
|
||||
|
||||
def test_failed_build_is_not_cached(cached_report) -> None:
|
||||
handler, engine, folder = cached_report
|
||||
handler.fail_once = True
|
||||
with pytest.raises(RuntimeError):
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert list(folder.glob("*.json")) == []
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert handler.calls == 2
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert handler.calls == 2
|
||||
|
||||
|
||||
def test_report_cache_directory_prunes_old_files(cached_report) -> None:
|
||||
handler, engine, folder = cached_report
|
||||
stale = folder / "stale-probe.json"
|
||||
stale.write_text("{}", encoding="utf-8")
|
||||
old = time.time() - (3 * 24 * 60 * 60)
|
||||
os.utime(stale, (old, old))
|
||||
build_professional_report_reference(handler, _body(), engine=engine)
|
||||
assert not stale.exists()
|
||||
Reference in New Issue
Block a user