feat: gate commercial external validation release

2026-07-17 14:32:56 +08:00
parent eeb4185bbc
commit 447d89f1da
5 changed files with 375 additions and 0 deletions
@@ -0,0 +1,161 @@
+# Commercial External Validation Release Implementation Plan
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Make the commercial repository's public external-validation evidence a versioned, hash-verified release gate while preserving every unresolved external-oracle boundary.
+
+**Architecture:** The existing public research reports remain the evidence payload. A small manifest records their expected SHA-256 digests and declared engine boundaries; a standalone Python gate validates file integrity and projects the VedAstro/JHora closure states into machine-readable output. The runtime-truth quality profile executes the gate so evidence drift blocks acceptance without requiring private raw artifacts or credentials.
+
+**Tech Stack:** Python 3 standard library, JSON, pytest, existing `scripts/run_quality_gate.py` profiles.
+
+### Task 1: Specify the public evidence release
+
+**Files:**
+- Create: `references/evidence_manifests/commercial_external_validation_release.v1.json`
+- Test: `tests/test_external_validation_release_gate.py`
+
+- [x] **Step 1: Write the failing manifest-contract test**
+
+```python
+def test_release_manifest_declares_public_assets_and_external_boundaries() -> None:
+    manifest = _manifest()
+    assert manifest["schema_version"] == 1
+    assert manifest["release_scope"] == "public_research_evidence_snapshot"
+    assert manifest["engines"]["PyJHora"]["status"] == "available"
+    assert manifest["engines"]["VedAstro"]["status"] == "blocked"
+    assert manifest["engines"]["JHora"]["official_raw_status"] != "verified"
+```
+
+- [x] **Step 2: Run the test to verify it fails**
+
+Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py::test_release_manifest_declares_public_assets_and_external_boundaries`
+
+Expected: FAIL because the manifest and `_manifest` helper do not exist.
+
+- [x] **Step 3: Add the versioned manifest**
+
+Record the eight already-versioned research reports, their SHA-256 digests, scope, and non-escalation boundaries. Record `PyJHora` and `jyotishganit` as locally available, `VedAstro` as blocked pending official replay closure, and JHora raw evidence as not verified.
+
+- [x] **Step 4: Run the manifest-contract test**
+
+Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py::test_release_manifest_declares_public_assets_and_external_boundaries`
+
+Expected: PASS.
+
+### Task 2: Implement integrity and boundary gate
+
+**Files:**
+- Create: `scripts/external_validation_release_gate.py`
+- Modify: `tests/test_external_validation_release_gate.py`
+
+- [x] **Step 1: Write failing gate behavior tests**
+
+```python
+def test_evaluate_manifest_accepts_current_public_release() -> None:
+    report = gate.evaluate_manifest(MANIFEST)
+    assert report["status"] == "pass"
+    assert report["summary"]["assets_verified"] == report["summary"]["assets_total"]
+    assert report["summary"]["production_tuning_allowed"] is False
+
+def test_evaluate_manifest_reports_digest_drift(tmp_path: Path) -> None:
+    manifest = _copy_manifest_with_one_bad_digest(tmp_path)
+    report = gate.evaluate_manifest(manifest)
+    assert report["status"] == "blocked"
+    assert report["assets"][0]["integrity"] == "mismatch"
+```
+
+- [x] **Step 2: Run tests to verify they fail**
+
+Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py -v`
+
+Expected: FAIL because `scripts.external_validation_release_gate` does not exist.
+
+- [x] **Step 3: Add the minimal gate**
+
+Implement `evaluate_manifest(path)` using `hashlib.sha256`; return JSON with per-asset existence/integrity, engine states, `production_tuning_allowed: false`, and `status: pass` only when every asset matches. Add CLI `--manifest`, `--format json`, and `--require-match`; `--require-match` returns non-zero for missing or mismatched assets only, not merely because VedAstro remains blocked.
+
+- [x] **Step 4: Run focused tests and CLI**
+
+Run:
+
+```bash
+python3 -m pytest -q tests/test_external_validation_release_gate.py
+python3 scripts/external_validation_release_gate.py --format json --require-match
+```
+
+Expected: tests PASS; CLI returns zero with `status: pass` and preserves `VedAstro: blocked` plus `production_tuning_allowed: false`.
+
+### Task 3: Wire the gate into runtime truth
+
+**Files:**
+- Modify: `scripts/run_quality_gate.py`
+- Modify: `tests/test_external_validation_release_gate.py`
+
+- [x] **Step 1: Write the failing quality-profile test**
+
+```python
+def test_runtime_truth_profile_runs_external_validation_release_gate() -> None:
+    text = (ROOT / "scripts" / "run_quality_gate.py").read_text(encoding="utf-8")
+    assert "external_validation_release_gate.py" in text
+```
+
+- [x] **Step 2: Run it to verify it fails**
+
+Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py::test_runtime_truth_profile_runs_external_validation_release_gate`
+
+Expected: FAIL because the release gate is not yet part of the quality gate.
+
+- [x] **Step 3: Add the runtime-truth command**
+
+Add the release-gate invocation to the runtime-truth command list with `--require-match`; retain the existing oracle collection semantics and do not convert blocked external engines into failures.
+
+- [x] **Step 4: Run runtime-truth and focused regression suite**
+
+Run:
+
+```bash
+python3 scripts/run_quality_gate.py --profile runtime-truth
+python3 -m pytest -q tests/test_external_validation_release_gate.py tests/test_oracle_closure_master_dashboard.py tests/test_three_engine_parity_replay_validator.py tests/test_cross_project_contract.py tests/test_cross_project_sync_status.py
+```
+
+Expected: PASS. The gate proves release integrity; output continues to state that prediction accuracy and production tuning remain blocked.
+
+### Task 4: Record the commercial release boundary
+
+**Files:**
+- Modify: `docs/superpowers/plans/2026-07-17-commercial-external-validation-release.md`
+
+- [x] **Step 1: Keep the gate commercial-only**
+
+The evidence files are already byte-identical research-derived public assets. The new manifest and validator are a commercial acceptance layer, so they are deliberately not added to the bidirectional calculation-contract policy or ledger. No private scratch, credentials, or raw oracle captures are added.
+
+- [x] **Step 2: Record the completed scope in this plan**
+
+This release verifies public evidence integrity only. It does not claim external oracle closure, prediction accuracy, or production-tuning authorization.
+
+- [x] **Step 3: Verify release hygiene and working tree**
+
+Run:
+
+```bash
+git diff --check
+git status --short --branch
+python3 scripts/scan_public_artifact_privacy.py --format json
+```
+
+Expected: no whitespace errors, no privacy findings, and only intended files modified before commit.
+
+- [ ] **Step 4: Commit and push**
+
+```bash
+git add docs/superpowers/plans/2026-07-17-commercial-external-validation-release.md \
+  references/evidence_manifests/commercial_external_validation_release.v1.json \
+  scripts/external_validation_release_gate.py \
+  scripts/run_quality_gate.py \
+  tests/test_external_validation_release_gate.py \
+  references/cross_project_contract/sync_policy.v1.json \
+  references/cross_project_contract/sync_ledger.json
+git commit -m "feat: gate commercial external validation release"
+git push origin codex/cross-project-contract
+```
+
+Expected: remote branch contains the integrity gate; `main` remains untouched pending merge review.
@@ -0,0 +1,80 @@
+{
+  "schema_version": 1,
+  "artifact_id": "commercial_external_validation_release",
+  "release_scope": "public_research_evidence_snapshot",
+  "generated_at": "2026-07-17T00:00:00Z",
+  "engines": {
+    "PyJHora": {
+      "status": "available",
+      "boundary": "Public parity reports are integrity-verified here; each scoped comparison retains its own replay boundary."
+    },
+    "jyotishganit": {
+      "status": "available",
+      "boundary": "Local engine availability is not a claim of external outcome accuracy."
+    },
+    "VedAstro": {
+      "status": "blocked",
+      "boundary": "Official external replay closure remains unavailable in this public release; no production tuning is authorized."
+    },
+    "JHora": {
+      "status": "blocked",
+      "official_raw_status": "not_collected",
+      "boundary": "No redistributable JHora desktop raw evidence packet is versioned in this public release."
+    }
+  },
+  "release_boundaries": {
+    "external_oracle_closure": false,
+    "prediction_accuracy_verified": false,
+    "production_tuning_allowed": false
+  },
+  "assets": [
+    {
+      "id": "oracle_closure_master_dashboard",
+      "path": "docs/research/oracle_closure_master_dashboard_latest.md",
+      "sha256": "2c020f0f499abc48a037cd33cc8d2ff2ec3170527793dba9d42792782d7d3868",
+      "scope": "target-set oracle closure boundary"
+    },
+    {
+      "id": "public_benchmark_dashboard",
+      "path": "docs/research/public_benchmark_dashboard_latest.md",
+      "sha256": "5733b610d763836f3e44d4f7baad7915224322b725014656123db4ebf0c1c3db",
+      "scope": "public benchmark coverage summary"
+    },
+    {
+      "id": "pyjhora_same_chart_parity",
+      "path": "docs/research/pyjhora_same_chart_parity_2026_07_12.md",
+      "sha256": "c786c5c1fde22d58e029340f0e332a4665587a1000be074a0f625735d3907b84",
+      "scope": "same-chart PyJHora parity"
+    },
+    {
+      "id": "pyjhora_varga_ashtakavarga_shadbala_parity",
+      "path": "docs/research/pyjhora_d2_d4_ashtakavarga_shadbala_parity_2026_07_15.md",
+      "sha256": "ffdfa08ecb244e7ca751152ca2b18b6952f337e378099c68416b9e9e44a64890",
+      "scope": "D2/D4, Ashtakavarga, and Shadbala PyJHora parity"
+    },
+    {
+      "id": "vedastro_parity_matrix",
+      "path": "docs/research/vedastro_parity_matrix_latest.json",
+      "sha256": "7be19440e491b62500964fa17c22adb5fc15b0c53af02e14dcbf4940b60ac328",
+      "scope": "VedAstro closure status matrix"
+    },
+    {
+      "id": "vedastro_fast_path_checklist",
+      "path": "docs/research/vedastro_fast_path_checklist_latest.json",
+      "sha256": "fca4af90a32e990fe8c94804d776b55e57c87a2e26801c34f2528faa1abd10b3",
+      "scope": "VedAstro official replay prerequisites"
+    },
+    {
+      "id": "vedastro_parity_matrix_rendered",
+      "path": "docs/research/vedastro_parity_matrix_latest.md",
+      "sha256": "a0546572c4f9809167839d675672c5d343101c748d6b9a04fc1c55593fdeadbc",
+      "scope": "human-readable VedAstro closure status"
+    },
+    {
+      "id": "vedastro_fast_path_rendered",
+      "path": "docs/research/vedastro_fast_path_checklist_latest.md",
+      "sha256": "7f5ff0242278eca7eacd15e005c15930d8bae6896c79c68547c3f0fbb9ba4a58",
+      "scope": "human-readable VedAstro replay prerequisites"
+    }
+  ]
+}
@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+"""Verify the commercial public external-validation evidence release."""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+from pathlib import Path
+
+
+ROOT = Path(__file__).resolve().parents[1]
+DEFAULT_MANIFEST = ROOT / "references" / "evidence_manifests" / "commercial_external_validation_release.v1.json"
+
+
+def sha256_file(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def evaluate_manifest(manifest_path: Path = DEFAULT_MANIFEST) -> dict:
+    manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
+    assets = []
+    for item in manifest["assets"]:
+        path = ROOT / item["path"]
+        observed = sha256_file(path) if path.is_file() else None
+        integrity = "verified" if observed == item["sha256"] else ("missing" if observed is None else "mismatch")
+        assets.append(
+            {
+                "id": item["id"],
+                "path": item["path"],
+                "scope": item["scope"],
+                "expected_sha256": item["sha256"],
+                "observed_sha256": observed,
+                "integrity": integrity,
+            }
+        )
+
+    verified = sum(item["integrity"] == "verified" for item in assets)
+    boundaries = manifest["release_boundaries"]
+    return {
+        "artifact_id": manifest["artifact_id"],
+        "release_scope": manifest["release_scope"],
+        "status": "pass" if verified == len(assets) else "blocked",
+        "assets": assets,
+        "engines": manifest["engines"],
+        "summary": {
+            "assets_total": len(assets),
+            "assets_verified": verified,
+            "external_oracle_closure": boundaries["external_oracle_closure"],
+            "prediction_accuracy_verified": boundaries["prediction_accuracy_verified"],
+            "production_tuning_allowed": boundaries["production_tuning_allowed"],
+        },
+        "boundary": "A passing release validates only versioned public evidence integrity. It does not close external oracles, verify prediction accuracy, or authorize production tuning.",
+    }
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--manifest", type=Path, default=DEFAULT_MANIFEST)
+    parser.add_argument("--format", choices=("text", "json"), default="text")
+    parser.add_argument("--require-match", action="store_true")
+    args = parser.parse_args()
+    report = evaluate_manifest(args.manifest)
+    if args.format == "json":
+        print(json.dumps(report, ensure_ascii=False, indent=2))
+    else:
+        print(f"external validation release: {report['status']}")
+        print(f"assets: {report['summary']['assets_verified']}/{report['summary']['assets_total']} verified")
+        print(f"VedAstro: {report['engines']['VedAstro']['status']}")
+        print(f"production_tuning_allowed: {report['summary']['production_tuning_allowed']}")
+    return 0 if report["status"] == "pass" or not args.require_match else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
@@ -512,6 +512,7 @@ def main() -> int:
            ROOT / "scripts" / "diagnose_external_engine_adapters.py",
            ROOT / "scripts" / "interpretation_source_runtime_coverage.py",
            ROOT / "scripts" / "sync_final_evidence_packet_status.py",
+            ROOT / "scripts" / "external_validation_release_gate.py",
        ]:
            py_compile.compile(str(target), doraise=True)
            print(f"compiled {target.relative_to(ROOT)}")
@@ -519,6 +520,7 @@ def main() -> int:
        run([PYTHON, "scripts/interpretation_source_inventory_gate.py"])
        run([PYTHON, "scripts/diagnose_vedastro_mode.py", "--json"])
        run([PYTHON, "scripts/diagnose_external_engine_adapters.py", "--json"])
+        run([PYTHON, "scripts/external_validation_release_gate.py", "--require-match"])
    else:
        compile_targets()
        validate_json_files()
@@ -0,0 +1,53 @@
+"""Regression coverage for the public external-validation release boundary."""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import scripts.external_validation_release_gate as gate
+
+
+ROOT = Path(__file__).resolve().parents[1]
+MANIFEST = ROOT / "references" / "evidence_manifests" / "commercial_external_validation_release.v1.json"
+
+
+def _manifest() -> dict:
+    return json.loads(MANIFEST.read_text(encoding="utf-8"))
+
+
+def _copy_manifest_with_one_bad_digest(tmp_path: Path) -> Path:
+    manifest = _manifest()
+    manifest["assets"][0]["sha256"] = "0" * 64
+    path = tmp_path / "release.json"
+    path.write_text(json.dumps(manifest), encoding="utf-8")
+    return path
+
+
+def test_release_manifest_declares_public_assets_and_external_boundaries() -> None:
+    manifest = _manifest()
+    assert manifest["schema_version"] == 1
+    assert manifest["release_scope"] == "public_research_evidence_snapshot"
+    assert manifest["engines"]["PyJHora"]["status"] == "available"
+    assert manifest["engines"]["jyotishganit"]["status"] == "available"
+    assert manifest["engines"]["VedAstro"]["status"] == "blocked"
+    assert manifest["engines"]["JHora"]["official_raw_status"] != "verified"
+
+
+def test_evaluate_manifest_accepts_current_public_release() -> None:
+    report = gate.evaluate_manifest(MANIFEST)
+    assert report["status"] == "pass"
+    assert report["summary"]["assets_verified"] == report["summary"]["assets_total"]
+    assert report["summary"]["production_tuning_allowed"] is False
+    assert report["engines"]["VedAstro"]["status"] == "blocked"
+
+
+def test_evaluate_manifest_reports_digest_drift(tmp_path: Path) -> None:
+    report = gate.evaluate_manifest(_copy_manifest_with_one_bad_digest(tmp_path))
+    assert report["status"] == "blocked"
+    assert report["assets"][0]["integrity"] == "mismatch"
+
+
+def test_runtime_truth_profile_runs_external_validation_release_gate() -> None:
+    text = (ROOT / "scripts" / "run_quality_gate.py").read_text(encoding="utf-8")
+    assert "external_validation_release_gate.py" in text