diff --git a/docs/tasks/PROGRESS-full-data-parity-1-fix-20261022.md b/docs/tasks/PROGRESS-full-data-parity-1-fix-20261022.md new file mode 100644 index 00000000..52b46e68 --- /dev/null +++ b/docs/tasks/PROGRESS-full-data-parity-1-fix-20261022.md @@ -0,0 +1,95 @@ +# PROGRESS：#91 A 段第 1 次打回（只修报告写法） + +- 分支：`codex/full-data-parity-1-20261021` +- 范围：修复单 J1–J4。B 段 T4–T10、#93 未做。 +- 状态：本地完成，未推送。 + +## 结论 + +0、1、9 号虚构盘的中英两版里，新六章除下面 4 节以外，每一节都至少有一行真实数据。这 4 节公开资料包没有对应模块，仍印「未判定」，第一行写明原因。 + +中文附录的年运表列名过滤在 200 张盘上没有丢掉任何列，所以没有补 `FIELD_ZH`。瑜伽「作用」的粘连过滤也没有删条目。瑜伽「组合」会把英文句子整格印成「未判定」：200 张盘全部中招，去重后 1030 条，源头不改。 + +## J1 十二节 + +读的是 `build_professional_report_reference_packet` 放进公开工作表的位置。0、1、9 号盘中英两版都由测试确认至少一行不是「未判定 / Not determined」。 + +| 节 | 公开包位置 | +| --- | --- | +| Vimsopaka 分盘贡献 | `strengths_and_scores.vimsopaka` 下直接是行星键（分数、分档） | +| Karakamsha | `advanced_systems.jaimini.karakamsha` 的星座、度数、宫主 | +| Darakaraka | `advanced_systems.jaimini.darakaraka` 的 `dk_planet`、`dk_sign`、`dk_house`、`dk_degree`、`d9_sign`、`d9_dignity` | +| D9 尊贵 | `divisional_and_special_charts.varga_full.D9_Navamsa`，行星是直接键，尊贵在 `_dignity` | +| 分盘映射 | 同上各分盘的 `Ascendant.sign`。盘名印成 `D9 Navamsa` 这种，不印带下划线的键。D81 / D108 / D144 不进这张表 | +| D30 状态 | `varga_full.D30_Trimsamsa` 与 `_dignity` | +| D30 输入 | `strengths_and_scores.shadbala.planets.*.sthana_bala.sapta_d30` | +| 行星状态输入 | `advanced_systems.avasthas..avasthas` 的 Jagrat / Deeptadi 状态，外加度数 | +| 定位星终点与循环 | `advanced_systems.final_dispositors`。自循环印主星，多星循环印「循环 / Cycle」 | +| Sudarshana 参考点 | `divisional_and_special_charts.sudarshana.raw.reference_points` | +| Bhava Chalit 宫心 | `divisional_and_special_charts.bhava_chalit.raw.boundaries.houses` | +| 本命特殊点公式依据 | `advanced_systems.sahams` 里带星座或黄经的点，印名称、星座、度数、黄经。不印公式轨迹 | +| 注明日期的行运 | `timing_and_predictive_systems.transit_multi_reference` 的 `transit_analysis` 与 `target_date`，星宿来自 `advanced_systems.nakshatra_dasha.transit_overlay` | + +## 四节白名单 + +公开报告资料包未带这些模块（只在被拿掉的原始阅读结果里）。每节第一行是「公开报告资料包未带此模块」/ `This module is not in the public report packet`，表内仍是「未判定」。没有为它们改引擎。 + +| 节 | 原因 | +| --- | --- | +| Navamsha 小运时长 | `navamsha_dasha` 没有复制进公开工作表 | +| 五鸟活动 | `pancha_pakshi` 没有复制进公开工作表 | +| Rashi Tulya Navamsa | `rashi_tulya_navamsa` 没有复制进公开工作表 | +| 当前多系统收敛 | `dasa_convergence` 的逐领域行没有复制进公开工作表。不再用 `convergence_top_domains` 充数 | + +## J2 / J3 + +`tests/test_full_data_parity_chapters.py` 的 `test_every_section_has_a_real_cell_except_the_four_public_gaps`：0、1、9 号盘中英两版，白名单外每一节至少有一个数据单元格不是「未判定 / Not determined」。表头和分隔行不算。白名单只有上表 4 节，每项带了原因。 + +`test_parity_chapters_go_through_local_conventions`：六章先拼进正文，再走 `_apply_local_report_conventions`。中文源字段「必然」渲染成该函数的结果「可能」；英文字段 `divorce` 渲染成 `relationship strain`。 + +## J4 丢弃统计 + +200 张虚构盘，PyJHora 已屏蔽。汉字为 0。计数在中文渲染里，同一条若被阅读表、资料附录和解释包各走一次，调用次数会高于条目数。条目数是每张盘内按规则和原文去重后再相加。 + +| 处 | 丢掉了什么 | 条目 | 调用 | 受影响盘 | +| --- | --- | --- | --- | --- | +| 年运表列名 | 无。过滤执行了 800 次（每盘 4 张表），没有列被 `FIELD_ZH` 拿掉 | 0 | 800 次过滤 | 0 | +| 瑜伽「作用」 | 无。粘连过滤没有删掉任何一条 | 0 | 0 | 0 | +| 瑜伽「组合」 | 非空的组合句整格变成「未判定」 | 1030 | 2496 | 200 | + +年运表优先列本来就有中文名，所以没有新列要补进 `FIELD_ZH`。`reading` 这类额外字段到不了这道过滤：优先列已经有值，回退列名单不会启用。 + +「作用」里的英文整句没有被删。粘连规则只删「汉字紧挨 in/and/or/from/planet/lord」、英文星座名，以及 `, :` / `in )`。BVR 规则的 `effects` 要么是纯中文，要么是纯英文，纯英文留在格子里。本单不改 `references/yoga_rules.json`。 + +「组合」变成「未判定」的源头： + +- `scripts/yoga_engine.py` 的 `_eval_semantic_condition`：没有中文模板时，用规则里的英文 `note` 当组合句。规则在 `references/yoga_rules.json`。按渲染调用次数：`bvr_krisanga_yoga` 501，`bvr_budha_aaditya_yoga` 315，`bvr_paasa_yoga` 282，`bvr_kedaara_yoga` 162，`bvr_vanchana_chora_bheethi_yoga` 144，`bvr_dehasthoulya_yoga` 125，`bvr_daama_yoga` 126，`bvr_kalpadruma_yoga` 108，`bvr_bahu_sthree_yoga` 102，`bvr_bhaarathi_yoga` 69，`bvr_grihanasa_yoga` 66，`bvr_asubha_yoga` 45，`bvr_subha_yoga` 45，`bvr_bandhu_pujya_yoga` 54，`bvr_guhyaroga_yoga` 27（句中有英文星座名），`bvr_vallaki_yoga` 18，`bvr_soola_yoga` 12，`bvr_rajalakshana_yoga` 9，`bvr_kalaanidhi_yoga` 3。 +- 没有 `rule_id` 的 283 次来自 `scripts/yogas_doshas.py`：`calc_raj_yogas` 与 `calc_dhana_yogas` 写出 `H*_Lord(...) in H*` 这种英文组合句。句子里的行星随盘变化，这里不列具体星。 + +这些句子是中文报告在用的英文字段。本单只计数，不改规则文件，也不改这两个函数。 + +## 阅读页上的英文星座名 + +`frontend/src/lib/personal-report-generation.ts` 的 `assembleReportDocumentV2`。它在组装 v2 报告文档时，把 D1 事实里的英文 `sign` 写进「本命基础」的 `narrative`，以及 `keyFactors` 里「第 N 宫落 {sign}」；宫内对象还是英文行星标识，两处句子加这一列，改一处星座映射盖不住。 + +阅读页 `frontend/src/components/personal-report/personal-report-page.tsx` 渲染的是 Markdown（`PersonalReportMarkdownView`）、盘图和事实表，不渲染这段 `natalFoundation`。`frontend/src/lib/report-public-projection.ts` 的 `foundationSection` 只是把它放到接口文档上。导出图盘列表的 `houseLine` 已经用中文星座名。本单不改前端，留给 Claude。 + +## 测试 + +解释器：`G:\Ferti\Jyotisha\.worktrees\agent-loop\.venv\Scripts\python.exe`。`PYTHONUTF8=1`。虚构盘助手已屏蔽 PyJHora。未跑快速门、全量 pytest、前端全量 `npm test`、`next build`。未改前端，未跑 `tsc` / `lint`。 + +| 命令 | 退出码 | +| --- | --- | +| `pytest tests/test_full_data_parity_chapters.py -q --tb=line` | 0（7 项，约 61 秒） | +| `pytest` 附录、英文冒烟、`test_report_english_edition`、`test_report_full_data_fix`、`test_full_data_domain_reader`、`test_python_hygiene_budget`、`test_api_server_growth_contract`、`test_runtime_jhora_import_scan`、`test_sidereal_nutation_frame`、`test_rashi_dashas_native`、`test_bhava_bala_formal`、`test_year_lord_blocked_no_fallback`、`test_year_lord_basis_label`、`test_repo_privacy_markers` | 0（172 项，约 925 秒） | +| `pytest tests/test_report_english_sweep.py -q --tb=line` | 0（200 张，约 1293 秒；汉字 0） | + +隐私标记在写完本文之后再跑一次。 + +## 偏离 + +- 没有推送，没有设上游，没有改标签，没有发 Gitea 评论。 +- 没有改 `scripts/jyotish_api_server.py`、`scripts/jaimini.py`、冻结打分文件、`references/yoga_rules.json`。没有提高 `ROOT_PY_BUDGET`。 +- 没有新的 BUG 编号。BUG-1416、BUG-1417 保持不变。 +- 年运列和瑜伽「作用」的实测丢弃是 0，所以没有补中文列名，也没有改瑜伽英文源。 +- 组合句的 1030 条是「非空原文被印成未判定」的盘内去重数，比「句中正好有 in/and/or/from/planet/lord」的范围稍宽，多出来的是纯英文说明句。调用次数 2496 是同一条被多处渲染重复计入。 diff --git a/docs/tasks/README.md b/docs/tasks/README.md index da9a1454..1149d9af 100644 --- a/docs/tasks/README.md +++ b/docs/tasks/README.md @@ -523,4 +523,4 @@ | `TASK-full-data-parity-3-20261022.md` | — | #93 完整数据版补齐（三）：「独立年度方法参考」正式环境调 PyJHora（子进程隔离；装上后其余输出逐字节不变，7 个换路径开关钉死）；输入默认用本地时间与正确年份（D6 待产品确认） | 待领取（等 #91 合入） | `codex/full-data-parity-3-20261022` | | `TASK-synastry-staging-fix-fix-20261022.md` | `PROGRESS-synastry-staging-fix-fix-20261022.md` | **#85 第 1 次打回**（T7～T11 主体通过）：新地址没登记进路由清单致快速门红；CHANGELOG 与 T11 矛盾；侧栏点亮测试是空断言 | 已合入 staging | `codex/synastry-staging-fix-20261021` | | `TASK-report-english-han-leak-fix-20261022.md` | `PROGRESS-report-english-han-leak-fix-20261022.md` | **#89 第 1 次打回**：改了冻结的校正打分文件 `jaimini.py`；测试的 PyJHora 屏蔽没传到子进程（测的不是生产那份报告，也是慢的原因）；缺英文时留空藏住漏洞 | 已合入 staging | `codex/report-english-han-leak-20261021` | -| `TASK-full-data-parity-1-fix-20261022.md` | `PROGRESS-full-data-parity-1-20261021.md` | **#91 A 段第 1 次打回**：新六章 69 节里 16 节在三张虚构盘上整节「未判定」，其中 12 节数据在资料包里、是读错了字段；测试抓不到整节未判定；新六章绕过了报告口径替换；中文附录悄悄丢列和瑜伽条目要量出来 | 待执行 | `codex/full-data-parity-1-20261021` | +| `TASK-full-data-parity-1-fix-20261022.md` | `PROGRESS-full-data-parity-1-fix-20261022.md` | **#91 A 段第 1 次打回**：新六章 69 节里 16 节在三张虚构盘上整节「未判定」，其中 12 节数据在资料包里、是读错了字段；测试抓不到整节未判定；新六章绕过了报告口径替换；中文附录悄悄丢列和瑜伽条目要量出来 | 本地完成，待验收 | `codex/full-data-parity-1-20261021` | diff --git a/scripts/pl9_full_data_export.py b/scripts/pl9_full_data_export.py index afe10e81..143c2c6f 100644 --- a/scripts/pl9_full_data_export.py +++ b/scripts/pl9_full_data_export.py @@ -2107,9 +2107,23 @@ def _render_pl9_ai_density_raw_data_markdown_en(packet: dict, language: str = "e ][:6] if language == "zh": try: - from scripts.report_render.appendix_language import FIELD_ZH + from scripts.report_render.appendix_language import ( + ANNUAL_COLUMN_NAMES_BEFORE_FIX, + FIELD_ZH, + note_annual_table, + note_dropped_annual_column, + ) except ModuleNotFoundError: # pragma: no cover - from report_render.appendix_language import FIELD_ZH + from report_render.appendix_language import ( + ANNUAL_COLUMN_NAMES_BEFORE_FIX, + FIELD_ZH, + note_annual_table, + note_dropped_annual_column, + ) + note_annual_table() + for field in keys: + if field not in ANNUAL_COLUMN_NAMES_BEFORE_FIX: + note_dropped_annual_column(field) keys = [field for field in keys if field in FIELD_ZH] if not keys: continue @@ -2365,6 +2379,12 @@ def _append_parity_chapters(markdown: str, packet: dict, language: str) -> str: def render_full_data_markdown(packet: dict) -> str: """Upstream `pl9_ai_density` branch at 23be1807, on this site's parity body.""" + if _pl9_report_language(packet if isinstance(packet, dict) else {}) != "en": + try: + from scripts.report_render.appendix_language import reset_appendix_drop_log + except ModuleNotFoundError: # pragma: no cover + from report_render.appendix_language import reset_appendix_drop_log + reset_appendix_drop_log() attach_full_data_interpretation, opening_focus_block = _interpretation_tools() if isinstance(packet, dict): attach_full_data_interpretation(packet) @@ -2396,8 +2416,8 @@ def render_full_data_markdown(packet: dict) -> str: english_markdown, _pl9_english_report_title(packet), ) - rendered = _apply_local_report_conventions(rendered, language="en") - return _append_parity_chapters(rendered, packet, "en") + rendered = _append_parity_chapters(rendered, packet, "en") + return _apply_local_report_conventions(rendered, language="en") zh_structured_appendix = re.sub( r"^#\s+.*?(?:\n{2,}|\n)", "", @@ -2418,8 +2438,8 @@ def render_full_data_markdown(packet: dict) -> str: + "\n\n\n" ) rendered = _sanitize_pl9_ai_density_markdown(zh_markdown, _pl9_public_report_title(packet)) - rendered = _apply_local_report_conventions(rendered, language="zh") - return _append_parity_chapters(rendered, packet, "zh") + rendered = _append_parity_chapters(rendered, packet, "zh") + return _apply_local_report_conventions(rendered, language="zh") def read_full_data_domain(packet: Mapping, domain: str) -> dict: diff --git a/scripts/report_render/appendix_language.py b/scripts/report_render/appendix_language.py index e30b2bfd..edf5e000 100644 --- a/scripts/report_render/appendix_language.py +++ b/scripts/report_render/appendix_language.py @@ -368,6 +368,63 @@ FIELD_ZH = { "months": "月数", } +# Names that already had a Chinese header before the #91 fix. +# The annual-table filter used to drop every other column. Counts use this set +# so a later Chinese name still shows up as a column the old filter removed. +ANNUAL_COLUMN_NAMES_BEFORE_FIX = frozenset(FIELD_ZH) + +_APPENDIX_DROPS: dict[str, dict[str, int]] = { + "annual_columns": {}, + "annual_tables": {}, + "effects": {}, + "combinations_undetermined": {}, + "combinations_replaced": {}, +} + + +def reset_appendix_drop_log() -> None: + """Clear the per-render drop counts. One Chinese render, then snapshot.""" + for bucket in _APPENDIX_DROPS.values(): + bucket.clear() + + +def snapshot_appendix_drop_log() -> dict[str, dict[str, int]]: + return {name: dict(bucket) for name, bucket in _APPENDIX_DROPS.items()} + + +def note_dropped_annual_column(field: str) -> None: + key = str(field) + bucket = _APPENDIX_DROPS["annual_columns"] + bucket[key] = bucket.get(key, 0) + 1 + + +def note_annual_table() -> None: + bucket = _APPENDIX_DROPS["annual_tables"] + bucket["seen"] = bucket.get("seen", 0) + 1 + + +def _drop_identity(item: dict, text: str) -> str: + parts = [ + str(item.get("rule_id") or ""), + str(item.get("source") or ""), + str(item.get("bvr_id") or ""), + str(item.get("name") or ""), + " ".join(str(text).split()), + ] + return " | ".join(parts)[:240] + + +def _note_effect(item: dict, text: str) -> None: + key = _drop_identity(item, text) + bucket = _APPENDIX_DROPS["effects"] + bucket[key] = bucket.get(key, 0) + 1 + + +def _note_combination(item: dict, text: str, kind: str) -> None: + key = _drop_identity(item, text) + bucket = _APPENDIX_DROPS[kind] + bucket[key] = bucket.get(key, 0) + 1 + def column_header(field: str, language: str) -> str: if language != "zh": @@ -591,12 +648,19 @@ def chinese_yoga_label(item: dict) -> str: def chinese_combination(item: dict) -> str: """Use the Chinese combination sentence. An English sentence is not translated word by word.""" combination = str(item.get("combination") or "").strip() - if combination and re.search(r"[\u4e00-\u9fff]", combination) and not _HAN_FUNCTION_RE.search(combination) and not SIGN_EN_RE.search(combination): + han = bool(re.search(r"[\u4e00-\u9fff]", combination)) + function = bool(_HAN_FUNCTION_RE.search(combination)) if combination else False + sign = bool(SIGN_EN_RE.search(combination)) if combination else False + if combination and han and not function and not sign: return combination planets = item.get("planets") if isinstance(item.get("planets"), list) else [] names = [planet_name(planet, "zh") for planet in planets if planet] if names: + if combination: + _note_combination(item, combination, "combinations_replaced") return "与".join(names) + if combination: + _note_combination(item, combination, "combinations_undetermined") return "未判定" @@ -610,7 +674,15 @@ def yoga_cells(item: dict, language: str) -> list[Any]: category_text = "未判定" strength = item.get("strength") or "-" effects = item.get("effects") if isinstance(item.get("effects"), list) else [] - kept = [str(effect) for effect in effects if str(effect).strip() and not _has_prose_glue(str(effect))] + kept = [] + for effect in effects: + text = str(effect).strip() + if not text: + continue + if _has_prose_glue(text): + _note_effect(item, text) + continue + kept.append(text) effects_text = "；".join(kept) or "-" return [chinese_yoga_label(item), category_text, strength, chinese_combination(item), effects_text] diff --git a/scripts/report_render/parity_chapters.py b/scripts/report_render/parity_chapters.py index de8da7b5..130027bf 100644 --- a/scripts/report_render/parity_chapters.py +++ b/scripts/report_render/parity_chapters.py @@ -470,6 +470,84 @@ def _rows_from(items: list[Any], language: str, columns: list[tuple[str, str]]) return _md_table(headers, rows) +_PLANETS = ("Sun", "Moon", "Mars", "Mercury", "Jupiter", "Venus", "Saturn", "Rahu", "Ketu") +_MISSING_MODULE = { + "zh": "公开报告资料包未带此模块", + "en": "This module is not in the public report packet", +} + + +def _missing_module(language: str, columns: list[str]) -> str: + """The public worksheet does not carry this module. Keep the undetermined row.""" + return _MISSING_MODULE[language] + "\n\n" + _blank(language, columns) + + +def _is_planet(value: Any) -> bool: + return isinstance(value, str) and value.strip() in _PLANETS + + +def _dignity_token(value: Any, language: str) -> Any: + if not isinstance(value, str): + return value + key = " ".join(value.strip().split()).lower() + if key == "neutral": + return "中性" if language == "zh" else "Neutral" + return { + "exalted": "exalted", + "debilitated": "debilitated", + "own sign": "own", + "own": "own", + "moolatrikona": "mooltrikona", + }.get(key, value) + + +def _named_planets(chart: Any) -> list[tuple[str, dict]]: + if not isinstance(chart, dict): + return [] + nested = chart.get("planets") + source = nested if isinstance(nested, dict) else chart + rows = [] + for name in _PLANETS: + item = source.get(name) + if isinstance(item, dict): + rows.append((name, item)) + return rows + + +def _varga_chart(packet: dict, *keys: str) -> dict: + varga = _first(packet, ("divisional_and_special_charts", "varga_full")) or {} + if not isinstance(varga, dict): + return {} + for key in keys: + chart = varga.get(key) + if isinstance(chart, dict): + return chart + return {} + + +def _varga_label(name: str, chart: dict) -> str: + meta = chart.get("_meta") if isinstance(chart.get("_meta"), dict) else {} + div = meta.get("div") + title = str(meta.get("name") or "") + if div and title and "_" not in title and not re.search(r"[\u4e00-\u9fff]", title): + return f"D{div} {title}".strip() + return " ".join(str(name).replace("_", " ").split()) + + +def _cycle_label(value: str, language: str) -> str | None: + body = value.split(":", 1)[1] + parts = [part.strip(" .") for part in body.replace("->", "|").replace("→", "|").split("|")] + names = [] + for part in parts: + if not _is_planet(part): + return None + names.append(planet_name(part, language)) + if len(names) < 2: + return None + prefix = "循环" if language == "zh" else "Cycle" + return prefix + " " + " → ".join(names) + + def _months(start: Any, end: Any) -> str | None: if not isinstance(start, str) or not isinstance(end, str): return None @@ -560,25 +638,8 @@ def _tribhagi_periods(packet: dict, language: str) -> str: def _navamsha(packet: dict, language: str) -> str: - module = _first(packet, ("timing_and_predictive_systems", "navamsha_dasha")) - periods = [] - if isinstance(module, dict): - majors = module.get("periods") or module.get("native_result", {}).get("periods") if isinstance(module.get("native_result"), dict) else module.get("periods") - if isinstance(module.get("native_result"), dict): - majors = module["native_result"].get("periods") or majors - for major in majors or []: - if not isinstance(major, dict): - continue - for child in major.get("antardasha") or []: - if not isinstance(child, dict): - continue - copied = dict(child) - copied["months"] = child.get("duration_months") or _months(child.get("start"), child.get("end")) - periods.append(copied) - if not periods: - note = "公开报告包没有带上 Navamsha 小运行。" if language == "zh" else "The public packet does not carry the Navamsha sub-period rows." - return note + "\n\n" + _blank(language, ["sign", "start", "end", "months"]) - return _rows_from(periods, language, [("sign", "sign"), ("start", "text"), ("end", "text"), ("months", "text")]) + del packet + return _missing_module(language, ["sign", "start", "end", "months"]) def _nakshatra_context(packet: dict, language: str) -> str: @@ -675,31 +736,8 @@ def _sandhi(packet: dict, language: str) -> str: def _convergence(packet: dict, language: str) -> str: - timing = _first(packet, ("ai_and_audit", "ai_prompt_pack", "evidence_snapshot", "timing")) or {} - pairs = timing.get("convergence_top_domains") if isinstance(timing, dict) else None - module = _first(packet, ("timing_and_predictive_systems", "dasa_convergence")) - activations = module.get("domain_activations") if isinstance(module, dict) else None - rows = [] - if isinstance(activations, dict) and activations: - for domain, info in activations.items(): - if not isinstance(info, dict): - continue - rows.append({ - "domain": domain, - "house": info.get("house"), - "level": info.get("convergence_level"), - "counted": info.get("system_count"), - }) - elif isinstance(pairs, list): - for item in pairs: - if isinstance(item, (list, tuple)) and len(item) >= 2: - rows.append({"domain": item[0], "level": item[1]}) - elif isinstance(item, dict): - rows.append({"domain": item.get("domain") or item.get("name"), "level": item.get("level") or item.get("convergence_level")}) - if not rows: - note = "公开报告包没有带上逐领域激活行。" if language == "zh" else "The public packet does not carry the per-domain activation rows." - return note + "\n\n" + _blank(language, ["domain", "level"]) - return _rows_from(rows, language, [("domain", "text"), ("house", "text"), ("level", "text"), ("counted", "text")]) + del packet + return _missing_module(language, ["domain", "level"]) def _chara(packet: dict, language: str) -> str: @@ -890,12 +928,30 @@ def _bhava(packet: dict, language: str) -> str: def _vimsopaka(packet: dict, language: str) -> str: module = _first(packet, ("strengths_and_scores", "vimsopaka")) or {} + nested = module.get("planets") if isinstance(module, dict) else None + planets = nested if isinstance(nested, dict) else module + bands = { + "High": ("High", "高"), + "Moderate": ("Moderate", "中"), + "Limited": ("Limited", "有限"), + "Low": ("Low", "低"), + } rows = [] - planets = module.get("planets") if isinstance(module, dict) else module if isinstance(planets, dict): for planet, item in planets.items(): - if isinstance(item, dict) and ("total_score" in item or "category" in item): - rows.append({"planet": planet, "score": item.get("total_score"), "category": item.get("category")}) + if not isinstance(item, dict) or ("total_score" not in item and "category" not in item): + continue + if planet not in _PLANETS and item.get("planet") not in _PLANETS: + continue + category = item.get("category") + pair = bands.get(str(category)) + if pair: + category = pair[1] if language == "zh" else pair[0] + rows.append({ + "planet": planet if planet in _PLANETS else item.get("planet"), + "score": item.get("total_score"), + "category": category, + }) elif isinstance(planets, list): rows = [item for item in planets if isinstance(item, dict)] return _rows_from(rows, language, [("planet", "planet"), ("score", "text"), ("category", "text")]) @@ -958,33 +1014,61 @@ def _prastara(packet: dict, language: str) -> str: return _rows_from(rows, language, [("planet", "planet"), ("total", "text")]) +def _avastha_state(item: dict) -> Any: + states = item.get("avasthas") if isinstance(item.get("avasthas"), dict) else {} + for key in ("Jagrat", "Deeptadi"): + entry = states.get(key) + if isinstance(entry, dict) and entry.get("state"): + return entry.get("state") + return item.get("jagradadi") or item.get("state") or item.get("avastha") + + def _avastha(packet: dict, language: str) -> str: module = _first(packet, ("advanced_systems", "avasthas")) or {} + nested = module.get("planets") if isinstance(module, dict) else None + planets = nested if isinstance(nested, dict) else module if isinstance(module, dict) else {} rows = [] - planets = module.get("planets") if isinstance(module, dict) else module - if isinstance(planets, dict): - for planet, item in planets.items(): - if isinstance(item, dict): - rows.append({"planet": planet, "status": item.get("jagradadi") or item.get("state") or item.get("avastha"), "degree": item.get("degree") or item.get("degree_in_sign")}) - elif isinstance(planets, list): - rows = [item for item in planets if isinstance(item, dict)] - return _rows_from(rows, language, [("planet", "planet"), ("status", "text"), ("degree", "text")]) + for planet, item in planets.items(): + if planet not in _PLANETS or not isinstance(item, dict): + continue + degree = item.get("degree") + if degree is None: + degree = item.get("degree_in_sign") + rows.append({ + "planet": planet, + "sign": item.get("sign"), + "status": _avastha_state(item), + "degree": degree, + }) + return _rows_from(rows, language, [("planet", "planet"), ("sign", "sign"), ("status", "text"), ("degree", "text")]) def _d30(packet: dict, language: str, source: bool) -> str: - module = _first(packet, ("advanced_systems", "trimshamsa_d30")) or _first(packet, ("divisional_and_special_charts", "varga_full", "D30")) - rows = [] - planets = module.get("planets") if isinstance(module, dict) else None - if isinstance(planets, dict): - for planet, item in planets.items(): - if not isinstance(item, dict): + if source: + rows = [] + for planet, item in _shadbala_planets(packet).items(): + if planet not in _PLANETS or not isinstance(item, dict): continue - if source: - rows.append({"planet": planet, "degree": item.get("degree_in_sign") or item.get("degree"), "sign": item.get("sign")}) - else: - rows.append({"planet": planet, "sign": item.get("sign"), "status": item.get("dignity") or item.get("status")}) - columns = [("planet", "planet"), ("degree", "text"), ("sign", "sign")] if source else [("planet", "planet"), ("sign", "sign"), ("status", "text")] - return _rows_from(rows, language, columns) + sthana = item.get("sthana_bala") if isinstance(item.get("sthana_bala"), dict) else {} + score = sthana.get("sapta_d30") + if isinstance(score, bool) or not isinstance(score, (int, float)): + continue + rows.append({"planet": planet, "score": score}) + return _rows_from(rows, language, [("planet", "planet"), ("score", "text")]) + chart = _varga_chart(packet, "D30_Trimsamsa", "D30", "Trimsamsa") + if not chart: + legacy = _first(packet, ("advanced_systems", "trimshamsa_d30")) + chart = legacy if isinstance(legacy, dict) else {} + dignity = chart.get("_dignity") if isinstance(chart.get("_dignity"), dict) else {} + rows = [] + for planet, item in _named_planets(chart): + status = dignity.get(planet) or item.get("dignity") or item.get("status") + rows.append({ + "planet": planet, + "sign": item.get("sign"), + "status": _dignity_token(status, language), + }) + return _rows_from(rows, language, [("planet", "planet"), ("sign", "sign"), ("status", "text")]) def _pushkara(packet: dict, language: str) -> str: @@ -1112,8 +1196,26 @@ def _chain_rows(module: Any, terminal: bool) -> list[dict]: def _dispositor(packet: dict, language: str, terminal: bool) -> str: - module = _first(packet, ("advanced_systems", "final_dispositors" if terminal else "dispositor_chains")) - return _rows_from(_chain_rows(module, terminal), language, [("planet", "planet"), ("sign", "sign"), ("lord", "planet")]) + if not terminal: + module = _first(packet, ("advanced_systems", "dispositor_chains")) + return _rows_from(_chain_rows(module, False), language, [("planet", "planet"), ("sign", "sign"), ("lord", "planet")]) + module = _first(packet, ("advanced_systems", "final_dispositors")) or {} + rows = [] + if isinstance(module, dict): + for planet, value in module.items(): + if planet not in _PLANETS: + continue + if _is_planet(value): + rows.append({"planet": planet, "lord": value}) + elif isinstance(value, str) and value.startswith("Cycle:"): + label = _cycle_label(value, language) + if label: + rows.append({"planet": planet, "value": label}) + elif isinstance(value, (dict, list)): + rows.extend(_chain_rows({planet: value}, True)) + if not rows: + rows = _chain_rows(module, True) + return _rows_from(rows, language, [("planet", "planet"), ("sign", "sign"), ("lord", "planet"), ("value", "text")]) def _aspects(packet: dict, language: str) -> str: @@ -1290,14 +1392,18 @@ def _saham_basis(packet: dict, language: str) -> str: for name, item in module.items(): if not isinstance(item, dict): continue - if "formula_status" not in item and "formula_variant" not in item and "formula_operands" not in item: + if item.get("sign") in (None, "") and not isinstance(item.get("longitude"), (int, float)): continue + label = item.get("name") + if not isinstance(label, str) or "_" in label or not label.strip(): + label = " ".join(str(name).replace("_", " ").split()) rows.append({ - "name": name, - "status": item.get("formula_status") or item.get("formula_variant"), - "value": item.get("base_longitude_before_exception") or item.get("longitude"), + "name": label, + "sign": item.get("sign"), + "degree": item.get("degree_in_sign"), + "longitude": item.get("longitude"), }) - return _rows_from(rows, language, [("name", "text"), ("status", "text"), ("value", "text")]) + return _rows_from(rows, language, [("name", "text"), ("sign", "sign"), ("degree", "text"), ("longitude", "text")]) def _special_points(packet: dict, language: str) -> str: @@ -1361,31 +1467,30 @@ def _one_point(packet: dict, language: str, key: str) -> str: item = jaimini.get(key) if isinstance(jaimini, dict) else None if isinstance(item, str): return _rows_from([{"planet": item}], language, [("planet", "planet")]) - if isinstance(item, dict): - return _rows_from([item], language, [("planet", "planet"), ("sign", "sign"), ("house", "text")]) - return _blank(language, ["planet", "sign"]) + if not isinstance(item, dict): + return _blank(language, ["planet", "sign"]) + if key == "karakamsha": + return _rows_from([{ + "sign": item.get("karakamsha_sign") or item.get("sign"), + "degree": item.get("karakamsha_degree") if "karakamsha_degree" in item else item.get("degree"), + "lord": item.get("karakamsha_lord") or item.get("lord") or item.get("planet"), + "house": item.get("house"), + }], language, [("sign", "sign"), ("degree", "text"), ("lord", "planet"), ("house", "text")]) + if key == "darakaraka": + return _rows_from([{ + "planet": item.get("dk_planet") or item.get("planet"), + "sign": item.get("dk_sign") or item.get("sign"), + "house": item.get("dk_house") if item.get("dk_house") is not None else item.get("house"), + "degree": item.get("dk_degree") if item.get("dk_degree") is not None else item.get("degree"), + "value": item.get("d9_sign"), + "dignity": _dignity_token(item.get("d9_dignity") or item.get("dignity"), language), + }], language, [("planet", "planet"), ("sign", "sign"), ("house", "text"), ("degree", "text"), ("value", "sign"), ("dignity", "text")]) + return _rows_from([item], language, [("planet", "planet"), ("sign", "sign"), ("house", "text")]) def _rashi_tulya(packet: dict, language: str) -> str: - module = _first(packet, ("advanced_systems", "rashi_tulya_navamsa")) or {} - analysis = module.get("analysis") if isinstance(module, dict) else module - if not isinstance(analysis, dict): - return _blank(language, ["name", "value"]) - labels = { - "lagna_sign": "上升星座" if language == "zh" else "Ascendant sign", - "strength_score": "力量分" if language == "zh" else "Strength score", - "weakness_score": "薄弱分" if language == "zh" else "Weakness score", - } - rows = [] - for key, label in labels.items(): - if key in analysis: - rows.append({"name": label, "value": analysis.get(key)}) - planets = analysis.get("planets") - if isinstance(planets, dict): - for planet, item in planets.items(): - sign = item.get("sign") if isinstance(item, dict) else item - rows.append({"name": planet, "value": sign}) - return _rows_from(rows, language, [("name", "text"), ("value", "text")]) + del packet + return _missing_module(language, ["name", "value"]) def _moon(packet: dict, language: str) -> str: @@ -1403,7 +1508,10 @@ def _moon(packet: dict, language: str) -> str: def _sudarshana(packet: dict, language: str) -> str: module = _first(packet, ("divisional_and_special_charts", "sudarshana")) or {} - points = module.get("reference_points") if isinstance(module, dict) else {} + raw = module.get("raw") if isinstance(module, dict) and isinstance(module.get("raw"), dict) else {} + points = raw.get("reference_points") if isinstance(raw.get("reference_points"), dict) else None + if not isinstance(points, dict) and isinstance(module, dict): + points = module.get("reference_points") rows = [] labels = { "ascendant_lagna": ("Ascendant reference", "出生上升参考点"), @@ -1419,16 +1527,37 @@ def _sudarshana(packet: dict, language: str) -> str: def _transit(packet: dict, language: str) -> str: - module = _first(packet, ("timing_and_predictive_systems", "transit_positions")) or _first(packet, ("advanced_systems", "transit_positions")) - if not isinstance(module, dict): - module = {} - planets = module.get("planets") or module.get("positions") or module + multi = _first(packet, ("timing_and_predictive_systems", "transit_multi_reference")) or {} + analysis = multi.get("transit_analysis") if isinstance(multi, dict) else {} + when = multi.get("target_date") if isinstance(multi, dict) else None + overlay_root = _first(packet, ("advanced_systems", "nakshatra_dasha")) or {} + overlay = overlay_root.get("transit_overlay") if isinstance(overlay_root, dict) else {} + if not isinstance(analysis, dict): + analysis = {} + if not isinstance(overlay, dict): + overlay = {} + order = [planet for planet in (*analysis.keys(), *overlay.keys()) if planet in _PLANETS] + seen = [] + for planet in order: + if planet not in seen: + seen.append(planet) rows = [] - if isinstance(planets, dict): - for planet, item in planets.items(): - if isinstance(item, dict): - rows.append({"planet": planet, "sign": item.get("sign"), "date": item.get("date") or module.get("date")}) - return _rows_from(rows, language, [("planet", "planet"), ("sign", "sign"), ("date", "text")]) + for planet in seen: + item = analysis.get(planet) if isinstance(analysis.get(planet), dict) else {} + over = overlay.get(planet) if isinstance(overlay.get(planet), dict) else {} + nakshatra = over.get("transit_nakshatra") + if isinstance(nakshatra, str) and "_" in nakshatra: + nakshatra = nakshatra.replace("_", " ") + if not isinstance(nakshatra, str): + nakshatra = None + rows.append({ + "planet": planet, + "sign": item.get("sign"), + "degree": item.get("degree_in_sign"), + "date": when if isinstance(when, (str, int, float)) else item.get("date"), + "nakshatra": nakshatra, + }) + return _rows_from(rows, language, [("planet", "planet"), ("sign", "sign"), ("degree", "text"), ("date", "text"), ("nakshatra", "text")]) def _multi_transit(packet: dict, language: str) -> str: @@ -1465,19 +1594,15 @@ def _kp(packet: dict, language: str) -> str: def _d9(packet: dict, language: str) -> str: - varga = _first(packet, ("divisional_and_special_charts", "varga_full")) or {} - chart = None - if isinstance(varga, dict): - for key in ("D9", "d9", "Navamsa", "navamsa"): - if isinstance(varga.get(key), dict): - chart = varga[key] - break - planets = chart.get("planets") if isinstance(chart, dict) else None + chart = _varga_chart(packet, "D9_Navamsa", "D9", "d9", "Navamsa", "navamsa") + dignity = chart.get("_dignity") if isinstance(chart.get("_dignity"), dict) else {} rows = [] - if isinstance(planets, dict): - for planet, item in planets.items(): - if isinstance(item, dict): - rows.append({"planet": planet, "sign": item.get("sign"), "dignity": item.get("dignity") or item.get("status")}) + for planet, item in _named_planets(chart): + rows.append({ + "planet": planet, + "sign": item.get("sign"), + "dignity": _dignity_token(dignity.get(planet) or item.get("dignity") or item.get("status"), language), + }) return _rows_from(rows, language, [("planet", "planet"), ("sign", "sign"), ("dignity", "text")]) @@ -1486,13 +1611,19 @@ def _mapping(packet: dict, language: str) -> str: rows = [] if isinstance(varga, dict): for name, chart in varga.items(): + if not isinstance(chart, dict): + continue if any(token in str(name).upper() for token in ("D81", "D108", "D144")): continue - sign = None - if isinstance(chart, dict): - asc = chart.get("ascendant") if isinstance(chart.get("ascendant"), dict) else {} - sign = asc.get("sign") or chart.get("asc_sign") or chart.get("lagna_sign") - rows.append({"name": name, "sign": sign}) + asc = chart.get("Ascendant") if isinstance(chart.get("Ascendant"), dict) else {} + if not asc and isinstance(chart.get("ascendant"), dict): + asc = chart.get("ascendant") or {} + sign = asc.get("sign") or chart.get("asc_sign") or chart.get("lagna_sign") + if sign in (None, "") and isinstance(asc.get("sign_idx"), int): + sign = asc.get("sign_idx") + if sign in (None, ""): + continue + rows.append({"name": _varga_label(str(name), chart), "sign": sign}) return _rows_from(rows, language, [("name", "text"), ("sign", "sign")]) @@ -1515,7 +1646,9 @@ def _chalit_planets(packet: dict, language: str) -> str: def _chalit_centres(packet: dict, language: str) -> str: raw = _bhava_raw(packet) boundaries = raw.get("boundaries") if isinstance(raw.get("boundaries"), dict) else {} - details = boundaries.get("houses_detail") if isinstance(boundaries, dict) else None + details = boundaries.get("houses_detail") if isinstance(boundaries.get("houses_detail"), list) else None + if not details: + details = boundaries.get("houses") if isinstance(boundaries.get("houses"), list) else [] rows = [] for item in details or []: if isinstance(item, dict): @@ -1625,7 +1758,7 @@ def _render_named(packet: dict, language: str, title: str) -> str: "Tara-Chandra Matrix": lambda: _tara(packet, language), "Natal Tithi Lord": lambda: _tithi(packet, language), "Birth Calendar": lambda: _panchanga(packet, language), - "Pancha Pakshi": lambda: _simple_module(packet, ("advanced_systems", "pancha_pakshi"), language, [("name", "text"), ("value", "text"), ("status", "text")]), + "Pancha Pakshi": lambda: _missing_module(language, ["name", "value", "status"]), "Natal Vivah Saham": lambda: _vivah(packet, language), "Natal Saham Formula Basis": lambda: _saham_basis(packet, language), "Arudha / UL / A10": lambda: _special_points(packet, language), diff --git a/tests/report_english_fictional.py b/tests/report_english_fictional.py index 131c7088..78ef53a7 100644 --- a/tests/report_english_fictional.py +++ b/tests/report_english_fictional.py @@ -278,6 +278,9 @@ def render_english(index: int) -> dict: edition = _english_edition(packet, True) chinese = render_full_data_markdown(packet) gates = chinese_prose_gates(chinese) + from scripts.report_render.appendix_language import snapshot_appendix_drop_log + + drops = snapshot_appendix_drop_log() markdown = edition.get("markdown_en") or "" flags = _flags_from_packet(packet) applicability = edition.get("reader_dasha_applicability_en") or "" @@ -298,6 +301,7 @@ def render_english(index: int) -> dict: "missing": rendered.count("[translation missing:"), "zh_english_signs": gates["english_signs"], "zh_han_function_glue": gates["han_function_glue"], + "appendix_drops": drops, **_guard_snapshot(), **flags, } diff --git a/tests/test_full_data_parity_chapters.py b/tests/test_full_data_parity_chapters.py index a00bb832..3c952f9b 100644 --- a/tests/test_full_data_parity_chapters.py +++ b/tests/test_full_data_parity_chapters.py @@ -6,6 +6,7 @@ are separate headings. Fictional charts only. from __future__ import annotations +import copy import re import pytest @@ -142,6 +143,108 @@ def test_three_fictional_charts_render_the_same_chapters(fictional_reports: dict assert "blocked" not in zh.lower() +_UNDETERMINED = {"zh": "未判定", "en": "Not determined"} +_GAP_LINE = { + "zh": "公开报告资料包未带此模块", + "en": "This module is not in the public report packet", +} +# These four modules are absent from the public worksheet. The reason is why +# the section may stay undetermined. Anything else must show a real cell. +_PACKET_GAPS = { + "Navamsha AD Duration": "navamsha_dasha is computed on the reading but is not copied into the public worksheet", + "Pancha Pakshi": "pancha_pakshi is not copied into the public worksheet", + "Rashi Tulya Navamsa": "rashi_tulya_navamsa is not copied into the public worksheet", + "Current Dasha Convergence": "dasa_convergence per-domain rows are not copied into the public worksheet", +} + + +def _table_blocks(body: str) -> list[list[list[str]]]: + blocks = [] + current: list[str] = [] + for line in body.splitlines(): + if line.startswith("|"): + current.append(line) + continue + if current: + blocks.append(current) + current = [] + if current: + blocks.append(current) + parsed = [] + for block in blocks: + rows = [] + for index, line in enumerate(block): + cells = [cell.strip() for cell in line.strip().strip("|").split("|")] + if cells and all(set(cell) <= set("-: ") and cell for cell in cells): + continue + if index == 0: + continue + rows.append(cells) + parsed.append(rows) + return parsed + + +def _section_has_real_cell(body: str, undetermined: str) -> bool: + blocks = _table_blocks(body) + if blocks: + return any( + cell and cell != undetermined + for rows in blocks + for row in rows + for cell in row + ) + prose = [line.strip() for line in body.splitlines() if line.strip()] + return any(line and line != undetermined for line in prose) + + +def test_every_section_has_a_real_cell_except_the_four_public_gaps(fictional_reports: dict[int, tuple[str, str]]) -> None: + titles = dict(SECTIONS) + for reason in _PACKET_GAPS.values(): + assert reason + for index, (chinese, english) in fictional_reports.items(): + for language, markdown in (("zh", chinese), ("en", english)): + chapter = _chapter_text(markdown, language) + for title_en, title_zh in SECTIONS: + heading = title_zh if language == "zh" else title_en + body = _section_body(chapter, heading) + if title_en in _PACKET_GAPS: + first = next(line.strip() for line in body.splitlines() if line.strip()) + assert first == _GAP_LINE[language], f"{index} {heading}" + continue + assert _section_has_real_cell(body, _UNDETERMINED[language]), f"{index} {heading}" + + +def test_parity_chapters_go_through_local_conventions() -> None: + from scripts.pl9_full_data_export import _apply_local_report_conventions + + packet = { + "report_language": "zh", + "worksheets": { + "strengths_and_scores": { + "vimsopaka": { + "Sun": {"planet": "Sun", "total_score": 12.5, "category": "必然"}, + } + } + }, + } + chinese = render_full_data_markdown(packet) + zh_body = _section_body(chinese, "Vimsopaka 分盘贡献") + zh_replaced = _apply_local_report_conventions("必然", language="zh") + assert zh_replaced == "可能" + assert zh_replaced in zh_body + assert "必然" not in zh_body + + english_packet = copy.deepcopy(packet) + english_packet["report_language"] = "en" + english_packet["worksheets"]["strengths_and_scores"]["vimsopaka"]["Sun"]["category"] = "divorce" + english = render_full_data_markdown(english_packet) + en_body = _section_body(english, "Vimsopaka Contribution") + en_replaced = _apply_local_report_conventions("divorce", language="en") + assert en_replaced == "relationship strain" + assert en_replaced in en_body + assert "divorce" not in en_body.lower() + + def test_chart_nine_partial_kaal_and_friendship(fictional_reports: dict[int, tuple[str, str]]) -> None: chinese, english = fictional_reports[9] assert _KAAL_ZH.search(chinese) diff --git a/tests/test_report_english_sweep.py b/tests/test_report_english_sweep.py index 4b3d4309..895438ce 100644 --- a/tests/test_report_english_sweep.py +++ b/tests/test_report_english_sweep.py @@ -3,24 +3,109 @@ This file is not on the quick gate. Windows timing is recorded in docs/tasks/PROGRESS-report-english-han-leak-20261021.md. Set REPORT_ENGLISH_SWEEP_DUMP to a directory to write the English markdown for -the acceptance pass. Workers stay at 4. +the acceptance pass. Workers stay at 4. Set J4_DROP_STATS to a json path to +write the Chinese-appendix drop counts from the same run. """ from __future__ import annotations +import json import os from concurrent.futures import ProcessPoolExecutor, as_completed +from pathlib import Path from tests.report_english_fictional import CHART_COUNT, init_english_worker, render_english, row_problem +def _j4_summary(rows: list[dict]) -> dict: + annual_names: dict[str, int] = {} + annual_charts = [] + annual_tables = 0 + effect_entries = 0 + effect_calls = 0 + effect_charts = [] + effect_keys: dict[str, int] = {} + combination_entries = 0 + combination_calls = 0 + combination_charts = [] + combination_keys: dict[str, int] = {} + replaced_entries = 0 + replaced_charts = [] + replaced_keys: dict[str, int] = {} + for row in rows: + drops = row.get("appendix_drops") if isinstance(row.get("appendix_drops"), dict) else {} + annual = drops.get("annual_columns") if isinstance(drops.get("annual_columns"), dict) else {} + tables = drops.get("annual_tables") if isinstance(drops.get("annual_tables"), dict) else {} + effects = drops.get("effects") if isinstance(drops.get("effects"), dict) else {} + combinations = drops.get("combinations_undetermined") if isinstance(drops.get("combinations_undetermined"), dict) else {} + replaced = drops.get("combinations_replaced") if isinstance(drops.get("combinations_replaced"), dict) else {} + if annual: + annual_charts.append(row.get("i")) + for name, count in annual.items(): + annual_names[name] = annual_names.get(name, 0) + int(count) + annual_tables += int(tables.get("seen") or 0) + if effects: + effect_charts.append(row.get("i")) + effect_entries += len(effects) + effect_calls += sum(int(count) for count in effects.values()) + for key, count in effects.items(): + effect_keys[key] = effect_keys.get(key, 0) + int(count) + if combinations: + combination_charts.append(row.get("i")) + combination_entries += len(combinations) + combination_calls += sum(int(count) for count in combinations.values()) + for key, count in combinations.items(): + combination_keys[key] = combination_keys.get(key, 0) + int(count) + if replaced: + replaced_charts.append(row.get("i")) + replaced_entries += len(replaced) + for key, count in replaced.items(): + replaced_keys[key] = replaced_keys.get(key, 0) + int(count) + return { + "charts": len(rows), + "annual_columns": annual_names, + "annual_chart_count": len(annual_charts), + "annual_tables_seen": annual_tables, + "effect_entries": effect_entries, + "effect_calls": effect_calls, + "effect_chart_count": len(effect_charts), + "effect_keys": effect_keys, + "combination_undetermined_entries": combination_entries, + "combination_undetermined_calls": combination_calls, + "combination_undetermined_chart_count": len(combination_charts), + "combination_undetermined_keys": combination_keys, + "combination_replaced_entries": replaced_entries, + "combination_replaced_chart_count": len(replaced_charts), + "combination_replaced_keys": replaced_keys, + } + + def test_two_hundred_fictional_charts_have_clean_english() -> None: workers = min(4, os.cpu_count() or 1, CHART_COUNT) failures = [] + rows = [] with ProcessPoolExecutor(max_workers=workers, initializer=init_english_worker) as pool: futures = [pool.submit(render_english, index) for index in range(CHART_COUNT)] for future in as_completed(futures): row = future.result() + rows.append(row) problem = row_problem(row) if problem: failures.append(problem) + summary = _j4_summary(rows) + print("J4_APPENDIX_DROPS " + json.dumps({ + "annual_columns": summary["annual_columns"], + "annual_chart_count": summary["annual_chart_count"], + "annual_tables_seen": summary["annual_tables_seen"], + "effect_entries": summary["effect_entries"], + "effect_calls": summary["effect_calls"], + "effect_chart_count": summary["effect_chart_count"], + "combination_undetermined_entries": summary["combination_undetermined_entries"], + "combination_undetermined_calls": summary["combination_undetermined_calls"], + "combination_undetermined_chart_count": summary["combination_undetermined_chart_count"], + "combination_replaced_entries": summary["combination_replaced_entries"], + "combination_replaced_chart_count": summary["combination_replaced_chart_count"], + }, ensure_ascii=False)) + stats_path = os.environ.get("J4_DROP_STATS") + if stats_path: + Path(stats_path).write_text(json.dumps(summary, ensure_ascii=False, indent=2), encoding="utf-8") assert failures == []