Classify full interpretation source inventory

docs/research/interpretation_source_full_classification_2026_07_02.md

@@ -0,0 +1,60 @@
+# Interpretation Source Full Classification - 2026-07-02
+
+This report is generated from `scripts/interpretation_source_inventory_gate.py`.
+It classifies local interpretation/rule/case/template/evidence candidates without
+promoting drafts into runtime truth.
+
+## Summary
+
+| Metric | Count |
+|---|---:|
+| Runtime source refs already wired | 22 |
+| Indexed-only refs | 3 |
+| Full candidate pool | 948 |
+| Unclassified candidates | 0 |
+| Priority 1 candidates | 264 |
+| Priority 2 candidates | 101 |
+| Priority 3 candidates | 558 |
+
+## Classification Counts
+
+| Classification | Count |
+|---|---:|
+| runtime_reference_layer | 22 |
+| indexed_reference_layer | 3 |
+| reference_candidate | 171 |
+| real_case_calibration | 5 |
+| open_source_reference | 102 |
+| oracle_artifact | 50 |
+| benchmark_evidence | 33 |
+| template_asset | 4 |
+| frontend_surface | 31 |
+| research_governance | 412 |
+| quarantined_draft | 87 |
+| project_governance | 28 |
+
+## Promotion Status Counts
+
+| Promotion Status | Count |
+|---|---:|
+| already_wired | 22 |
+| indexed | 3 |
+| reference_layer_candidate | 278 |
+| real/source evidence only buckets combined | 83 |
+| governance_or_history | 440 |
+| not_truth_source | 87 |
+| product_surface | 31 |
+| template_reference | 4 |
+
+## Priority Policy
+
+- `priority_1`: review first. This includes `references/`, `real_case_studies/`, `rishi-ai-mcp`, and `vedic-astro-skills`.
+- `priority_2`: validation/template/oracle evidence. These can support boundaries and tests but are not direct interpretive truth.
+- `priority_3`: research history, frontend surfaces, project governance, and quarantined drafts. These are indexed so they are not forgotten, but they must not be silently promoted.
+- `runtime`: already wired or indexed by the active source pack.
+
+## Boundary
+
+This is a classification map, not a promotion decision. A file classified as
+`reference_layer_candidate` still needs source review, license review when
+applicable, and focused tests before it can become a runtime source.

progress.md

@@ -738,3 +738,17 @@
 - 最小实现：`mcp_server.py` 新增 `_existing_interpretation_source_pack()`，把现有 `interpretation_template_registry`、P1-P12、house framework、Raman/BPHS、MEVG、真实案例 checklist、前端 planet-house-details 作为只读 evidence pack 挂入 career/relationship/finance strict workflow；`technique_audit_summary` 新增 `interpretation_source_pack`、`mevg_global_web_evidence`、`real_case_calibration` 三行。
 - Prompt pack：`scripts/jyotish_engine.py` 的 `technique_audit_table` 新增 `Interpretation Source Pack`、`MEVG / Global Web Evidence`、`Real Case Calibration`；`retrieval_plan.local_reference_docs` 显式列出对应本地资料路径。
 - 红绿验证：5 条新增/修改聚焦测试通过；随后 `tests/test_mcp_strict_workflow_career.py tests/test_mcp_strict_workflow_relationship.py tests/test_mcp_strict_workflow_finance.py tests/test_mcp_strict_workflow_functional_layer.py -q` 通过，合计 86 项；`validate_interpretation_templates.py --format json` 仍为 `valid=true`、`template_count=11`。
+
+## 2026-07-02T14:05:00+08:00 - 全量解释资料分级审计启动
+
+- 用户要求把此前机器审计发现的 922 个未分级候选资料逐批分类。
+- 范围优先级：先处理 references/、references/real_case_studies/、references/open_source_sources/rishi-ai-mcp、references/open_source_sources/vedic-astro-skills；普通 docs/research 流水账只分桶，不升格。
+- 实施策略：扩展 interpretation_source_inventory_gate，生成可复现 full_classification 报告；用测试确保重点层被分级、local_drafts/research 流水账不会误进 primary truth。
+
+## 2026-07-02T14:18:00+08:00 - 全量解释资料分级审计完成第一版
+
+- 扩展 `scripts/interpretation_source_inventory_gate.py`：现在输出 `full_classification`，对解释/规则/案例/模板/证据候选逐项给出 classification、priority、promotion_status 与 reason。
+- 当前候选池 `candidate_count=947`，`unclassified_candidate_count=0`；其中 runtime 25、priority_1 264、priority_2 101、priority_3 557。
+- 重点优先层已覆盖：`references/` -> reference_candidate，`references/real_case_studies/` -> real_case_calibration，`open_source_sources/rishi-ai-mcp` 与 `vedic-astro-skills` -> open_source_reference。
+- 普通研究流水账与本地草稿已降级：`docs/research` -> research_governance，`docs/research/local_drafts` -> quarantined_draft / not_truth_source。
+- 新增快照文档：`docs/research/interpretation_source_full_classification_2026_07_02.md`。

scripts/interpretation_source_inventory_gate.py

@@ -26,6 +26,78 @@ REQUIRED_LAYERS = [
     "quarantined_drafts",
 ]
 
+SCAN_ROOTS = [
+    "references",
+    "docs",
+    "jyotish-app",
+    "assets",
+    "SKILL.md",
+    "AGENTS.md",
+]
+
+EXCLUDED_PARTS = {
+    ".git",
+    ".pytest_cache",
+    ".ruff_cache",
+    "__pycache__",
+    "node_modules",
+    "venv_vedastro",
+    "build",
+    "dist",
+    "scratch",
+}
+
+TEXT_EXTENSIONS = {".md", ".txt", ".json", ".js", ".ts", ".tsx", ".jsx", ".html", ".csv"}
+
+CANDIDATE_KEYWORDS = [
+    "interpret",
+    "解释",
+    "解读",
+    "reading",
+    "reader",
+    "rule",
+    "rules",
+    "规则",
+    "template",
+    "模板",
+    "case",
+    "案例",
+    "bphs",
+    "parashara",
+    "parasara",
+    "raman",
+    "jataka",
+    "saravali",
+    "phaladeepika",
+    "hora",
+    "house",
+    "宫",
+    "lord",
+    "主",
+    "yoga",
+    "saham",
+    "dasha",
+    "transit",
+    "career",
+    "profession",
+    "事业",
+    "wealth",
+    "finance",
+    "财富",
+    "marriage",
+    "relationship",
+    "婚",
+    "关系",
+    "timing",
+    "应期",
+    "event",
+    "预测",
+    "validation",
+    "校验",
+    "qa",
+    "workflow",
+]
+
 
 def build_report() -> dict[str, Any]:
     source_pack = _existing_interpretation_source_pack()
@@ -63,6 +135,7 @@ def build_report() -> dict[str, Any]:
         "summary": summary,
         "layers": layers,
         "runtime_source_refs": source_refs,
+        "full_classification": _build_full_classification(source_refs, layers),
         "failures": failures,
         "boundary": (
             "This gate validates explicit source inventory wiring. It does not promote drafts "
@@ -71,6 +144,175 @@ def build_report() -> dict[str, Any]:
     }
 
 
+def _relative(path: Path) -> str:
+    return str(path.relative_to(ROOT))
+
+
+def _should_skip(path: Path) -> bool:
+    return any(part in EXCLUDED_PARTS for part in path.parts)
+
+
+def _iter_candidate_files() -> list[Path]:
+    items: list[Path] = []
+    for root_name in SCAN_ROOTS:
+        root = ROOT / root_name
+        if root.is_file():
+            items.append(root)
+        elif root.exists():
+            items.extend(path for path in root.rglob("*") if path.is_file())
+    return sorted(set(items))
+
+
+def _is_candidate(path: Path) -> bool:
+    if _should_skip(path) or path.suffix.lower() not in TEXT_EXTENSIONS:
+        return False
+    rel = _relative(path).lower()
+    name = path.name.lower()
+    if any(keyword.lower() in rel or keyword.lower() in name for keyword in CANDIDATE_KEYWORDS):
+        return True
+    if path.stat().st_size > 300_000:
+        return False
+    try:
+        text = path.read_text(encoding="utf-8", errors="ignore")[:4000].lower()
+    except OSError:
+        return False
+    return any(keyword.lower() in text for keyword in CANDIDATE_KEYWORDS)
+
+
+def _classify_candidate(path: str, runtime_source_refs: set[str], layer_refs: set[str]) -> dict[str, Any]:
+    if path in runtime_source_refs:
+        return {
+            "classification": "runtime_reference_layer",
+            "priority": "runtime",
+            "promotion_status": "already_wired",
+            "reason": "Already exposed through interpretation_source_pack.source_refs.",
+        }
+    if path in layer_refs:
+        return {
+            "classification": "indexed_reference_layer",
+            "priority": "runtime",
+            "promotion_status": "indexed",
+            "reason": "Indexed by the interpretation source inventory.",
+        }
+    if path.startswith("references/real_case_studies/"):
+        return {
+            "classification": "real_case_calibration",
+            "priority": "priority_1",
+            "promotion_status": "reference_layer_candidate",
+            "reason": "Real-case material should be reviewed before confidence calibration claims.",
+        }
+    if path.startswith("references/open_source_sources/rishi-ai-mcp/"):
+        return {
+            "classification": "open_source_reference",
+            "priority": "priority_1",
+            "promotion_status": "reference_layer_candidate",
+            "reason": "User-prioritized open-source skill/workflow corpus; must remain license-aware.",
+        }
+    if path.startswith("references/open_source_sources/vedic-astro-skills/"):
+        return {
+            "classification": "open_source_reference",
+            "priority": "priority_1",
+            "promotion_status": "reference_layer_candidate",
+            "reason": "User-prioritized open-source Jyotish skills corpus; classify before selective reuse.",
+        }
+    if path.startswith("references/open_source_sources/"):
+        return {
+            "classification": "open_source_reference",
+            "priority": "priority_2",
+            "promotion_status": "reference_layer_candidate",
+            "reason": "Open-source reference corpus; classify with license boundary before runtime use.",
+        }
+    if path.startswith("references/oracle/"):
+        return {
+            "classification": "oracle_artifact",
+            "priority": "priority_2",
+            "promotion_status": "oracle_evidence_only",
+            "reason": "External evidence artifact; do not convert into interpretive rule text.",
+        }
+    if path.startswith("references/"):
+        return {
+            "classification": "reference_candidate",
+            "priority": "priority_1",
+            "promotion_status": "reference_layer_candidate",
+            "reason": "Core references directory; review for source-of-truth or reference-layer promotion.",
+        }
+    if path.startswith("docs/research/local_drafts/"):
+        return {
+            "classification": "quarantined_draft",
+            "priority": "priority_3",
+            "promotion_status": "not_truth_source",
+            "reason": "Local draft; index for awareness but do not promote without explicit review.",
+        }
+    if path.startswith("docs/research/"):
+        return {
+            "classification": "research_governance",
+            "priority": "priority_3",
+            "promotion_status": "governance_or_history",
+            "reason": "Research/governance history; useful for audit, not direct interpretation truth.",
+        }
+    if path.startswith("docs/benchmark/") or path.startswith("benchmarks/") or path.startswith("benchmark/"):
+        return {
+            "classification": "benchmark_evidence",
+            "priority": "priority_2",
+            "promotion_status": "benchmark_evidence_only",
+            "reason": "Benchmark or evidence report; use for validation boundaries, not direct rules.",
+        }
+    if path.startswith("jyotish-app/"):
+        return {
+            "classification": "frontend_surface",
+            "priority": "priority_3",
+            "promotion_status": "product_surface",
+            "reason": "Frontend implementation or copy surface; classify separately from source truth.",
+        }
+    if path.startswith("assets/"):
+        return {
+            "classification": "template_asset",
+            "priority": "priority_2",
+            "promotion_status": "template_reference",
+            "reason": "Reusable template asset; can guide output structure but is not a rule source.",
+        }
+    return {
+        "classification": "project_governance",
+        "priority": "priority_3",
+        "promotion_status": "governance_or_history",
+        "reason": "Project-level planning/governance material.",
+    }
+
+
+def _build_full_classification(source_refs: list[str], layers: dict[str, Any]) -> dict[str, Any]:
+    runtime_source_refs = set(source_refs)
+    layer_refs: set[str] = set()
+    for layer in layers.values():
+        if isinstance(layer, dict):
+            refs = layer.get("source_refs")
+            if isinstance(refs, list):
+                layer_refs.update(str(ref) for ref in refs)
+
+    candidate_paths = [_relative(path) for path in _iter_candidate_files() if _is_candidate(path)]
+    by_path = {
+        path: _classify_candidate(path, runtime_source_refs, layer_refs)
+        for path in candidate_paths
+    }
+    classification_counts: dict[str, int] = {}
+    priority_bucket_counts: dict[str, int] = {}
+    promotion_status_counts: dict[str, int] = {}
+    for item in by_path.values():
+        classification_counts[item["classification"]] = classification_counts.get(item["classification"], 0) + 1
+        priority_bucket_counts[item["priority"]] = priority_bucket_counts.get(item["priority"], 0) + 1
+        promotion_status_counts[item["promotion_status"]] = promotion_status_counts.get(item["promotion_status"], 0) + 1
+    return {
+        "status": "classified",
+        "candidate_count": len(candidate_paths),
+        "classified_candidate_count": len(by_path),
+        "unclassified_candidate_count": 0,
+        "classification_counts": dict(sorted(classification_counts.items())),
+        "priority_bucket_counts": dict(sorted(priority_bucket_counts.items())),
+        "promotion_status_counts": dict(sorted(promotion_status_counts.items())),
+        "by_path": by_path,
+        "boundary": "Classification is a triage map; promotion still requires source review and tests.",
+    }
+
+
 def main() -> int:
     report = build_report()
     print(json.dumps(report, ensure_ascii=False, indent=2))

tests/test_interpretation_source_inventory_gate.py

@@ -55,3 +55,40 @@ def test_quality_gate_runs_interpretation_source_inventory_gate() -> None:
 
     assert '"scripts/interpretation_source_inventory_gate.py"' in quality_gate
     assert '[PYTHON, "scripts/interpretation_source_inventory_gate.py"]' in quality_gate
+
+
+def test_interpretation_source_inventory_gate_classifies_full_candidate_pool() -> None:
+    completed = subprocess.run(
+        [
+            sys.executable,
+            "scripts/interpretation_source_inventory_gate.py",
+        ],
+        cwd=ROOT,
+        text=True,
+        capture_output=True,
+        timeout=120,
+        check=False,
+    )
+
+    assert completed.returncode == 0, completed.stderr or completed.stdout
+    report = json.loads(completed.stdout)
+    classification = report["full_classification"]
+
+    assert classification["status"] == "classified"
+    assert classification["candidate_count"] >= 900
+    assert classification["unclassified_candidate_count"] == 0
+    assert classification["priority_bucket_counts"]["priority_1"] >= 50
+    assert classification["priority_bucket_counts"]["priority_2"] >= 50
+    assert classification["priority_bucket_counts"]["priority_3"] >= 100
+
+    by_path = classification["by_path"]
+    assert by_path["references/real_case_studies/vedicka/career-success-poverty-prosperity.md"]["classification"] == "real_case_calibration"
+    assert by_path["references/open_source_sources/rishi-ai-mcp/.agents/skills/career-analysis/SKILL.md"]["classification"] == "open_source_reference"
+    assert by_path["references/open_source_sources/vedic-astro-skills/codex/skills/vedic-core/resources/qa_rules.md"]["classification"] == "runtime_reference_layer"
+    assert by_path["references/advanced-techniques.md"]["classification"] == "reference_candidate"
+    assert by_path["docs/research/local_drafts/2026-06/antigravity_round31_api_completion_top50_2026_06_26.md"]["classification"] == "quarantined_draft"
+    assert by_path["docs/research/ACTIVE_FRONTS.md"]["classification"] == "research_governance"
+
+    assert by_path["references/open_source_sources/rishi-ai-mcp/.agents/skills/career-analysis/SKILL.md"]["priority"] == "priority_1"
+    assert by_path["references/real_case_studies/vedicka/career-success-poverty-prosperity.md"]["priority"] == "priority_1"
+    assert by_path["docs/research/local_drafts/2026-06/antigravity_round31_api_completion_top50_2026_06_26.md"]["promotion_status"] == "not_truth_source"