diff --git a/docs/superpowers/plans/2026-07-17-commercial-external-validation-release.md b/docs/superpowers/plans/2026-07-17-commercial-external-validation-release.md new file mode 100644 index 00000000..5602a6cd --- /dev/null +++ b/docs/superpowers/plans/2026-07-17-commercial-external-validation-release.md @@ -0,0 +1,161 @@ +# Commercial External Validation Release Implementation Plan +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make the commercial repository's public external-validation evidence a versioned, hash-verified release gate while preserving every unresolved external-oracle boundary. + +**Architecture:** The existing public research reports remain the evidence payload. A small manifest records their expected SHA-256 digests and declared engine boundaries; a standalone Python gate validates file integrity and projects the VedAstro/JHora closure states into machine-readable output. The runtime-truth quality profile executes the gate so evidence drift blocks acceptance without requiring private raw artifacts or credentials. + +**Tech Stack:** Python 3 standard library, JSON, pytest, existing `scripts/run_quality_gate.py` profiles. + +### Task 1: Specify the public evidence release + +**Files:** +- Create: `references/evidence_manifests/commercial_external_validation_release.v1.json` +- Test: `tests/test_external_validation_release_gate.py` + +- [x] **Step 1: Write the failing manifest-contract test** + +```python +def test_release_manifest_declares_public_assets_and_external_boundaries() -> None: + manifest = _manifest() + assert manifest["schema_version"] == 1 + assert manifest["release_scope"] == "public_research_evidence_snapshot" + assert manifest["engines"]["PyJHora"]["status"] == "available" + assert manifest["engines"]["VedAstro"]["status"] == "blocked" + assert manifest["engines"]["JHora"]["official_raw_status"] != "verified" +``` + +- [x] **Step 2: Run the test to verify it fails** + +Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py::test_release_manifest_declares_public_assets_and_external_boundaries` + +Expected: FAIL because the manifest and `_manifest` helper do not exist. + +- [x] **Step 3: Add the versioned manifest** + +Record the eight already-versioned research reports, their SHA-256 digests, scope, and non-escalation boundaries. Record `PyJHora` and `jyotishganit` as locally available, `VedAstro` as blocked pending official replay closure, and JHora raw evidence as not verified. + +- [x] **Step 4: Run the manifest-contract test** + +Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py::test_release_manifest_declares_public_assets_and_external_boundaries` + +Expected: PASS. + +### Task 2: Implement integrity and boundary gate + +**Files:** +- Create: `scripts/external_validation_release_gate.py` +- Modify: `tests/test_external_validation_release_gate.py` + +- [x] **Step 1: Write failing gate behavior tests** + +```python +def test_evaluate_manifest_accepts_current_public_release() -> None: + report = gate.evaluate_manifest(MANIFEST) + assert report["status"] == "pass" + assert report["summary"]["assets_verified"] == report["summary"]["assets_total"] + assert report["summary"]["production_tuning_allowed"] is False + +def test_evaluate_manifest_reports_digest_drift(tmp_path: Path) -> None: + manifest = _copy_manifest_with_one_bad_digest(tmp_path) + report = gate.evaluate_manifest(manifest) + assert report["status"] == "blocked" + assert report["assets"][0]["integrity"] == "mismatch" +``` + +- [x] **Step 2: Run tests to verify they fail** + +Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py -v` + +Expected: FAIL because `scripts.external_validation_release_gate` does not exist. + +- [x] **Step 3: Add the minimal gate** + +Implement `evaluate_manifest(path)` using `hashlib.sha256`; return JSON with per-asset existence/integrity, engine states, `production_tuning_allowed: false`, and `status: pass` only when every asset matches. Add CLI `--manifest`, `--format json`, and `--require-match`; `--require-match` returns non-zero for missing or mismatched assets only, not merely because VedAstro remains blocked. + +- [x] **Step 4: Run focused tests and CLI** + +Run: + +```bash +python3 -m pytest -q tests/test_external_validation_release_gate.py +python3 scripts/external_validation_release_gate.py --format json --require-match +``` + +Expected: tests PASS; CLI returns zero with `status: pass` and preserves `VedAstro: blocked` plus `production_tuning_allowed: false`. + +### Task 3: Wire the gate into runtime truth + +**Files:** +- Modify: `scripts/run_quality_gate.py` +- Modify: `tests/test_external_validation_release_gate.py` + +- [x] **Step 1: Write the failing quality-profile test** + +```python +def test_runtime_truth_profile_runs_external_validation_release_gate() -> None: + text = (ROOT / "scripts" / "run_quality_gate.py").read_text(encoding="utf-8") + assert "external_validation_release_gate.py" in text +``` + +- [x] **Step 2: Run it to verify it fails** + +Run: `python3 -m pytest -q tests/test_external_validation_release_gate.py::test_runtime_truth_profile_runs_external_validation_release_gate` + +Expected: FAIL because the release gate is not yet part of the quality gate. + +- [x] **Step 3: Add the runtime-truth command** + +Add the release-gate invocation to the runtime-truth command list with `--require-match`; retain the existing oracle collection semantics and do not convert blocked external engines into failures. + +- [x] **Step 4: Run runtime-truth and focused regression suite** + +Run: + +```bash +python3 scripts/run_quality_gate.py --profile runtime-truth +python3 -m pytest -q tests/test_external_validation_release_gate.py tests/test_oracle_closure_master_dashboard.py tests/test_three_engine_parity_replay_validator.py tests/test_cross_project_contract.py tests/test_cross_project_sync_status.py +``` + +Expected: PASS. The gate proves release integrity; output continues to state that prediction accuracy and production tuning remain blocked. + +### Task 4: Record the commercial release boundary + +**Files:** +- Modify: `docs/superpowers/plans/2026-07-17-commercial-external-validation-release.md` + +- [x] **Step 1: Keep the gate commercial-only** + +The evidence files are already byte-identical research-derived public assets. The new manifest and validator are a commercial acceptance layer, so they are deliberately not added to the bidirectional calculation-contract policy or ledger. No private scratch, credentials, or raw oracle captures are added. + +- [x] **Step 2: Record the completed scope in this plan** + +This release verifies public evidence integrity only. It does not claim external oracle closure, prediction accuracy, or production-tuning authorization. + +- [x] **Step 3: Verify release hygiene and working tree** + +Run: + +```bash +git diff --check +git status --short --branch +python3 scripts/scan_public_artifact_privacy.py --format json +``` + +Expected: no whitespace errors, no privacy findings, and only intended files modified before commit. + +- [ ] **Step 4: Commit and push** + +```bash +git add docs/superpowers/plans/2026-07-17-commercial-external-validation-release.md \ + references/evidence_manifests/commercial_external_validation_release.v1.json \ + scripts/external_validation_release_gate.py \ + scripts/run_quality_gate.py \ + tests/test_external_validation_release_gate.py \ + references/cross_project_contract/sync_policy.v1.json \ + references/cross_project_contract/sync_ledger.json +git commit -m "feat: gate commercial external validation release" +git push origin codex/cross-project-contract +``` + +Expected: remote branch contains the integrity gate; `main` remains untouched pending merge review. diff --git a/references/evidence_manifests/commercial_external_validation_release.v1.json b/references/evidence_manifests/commercial_external_validation_release.v1.json new file mode 100644 index 00000000..94e11196 --- /dev/null +++ b/references/evidence_manifests/commercial_external_validation_release.v1.json @@ -0,0 +1,80 @@ +{ + "schema_version": 1, + "artifact_id": "commercial_external_validation_release", + "release_scope": "public_research_evidence_snapshot", + "generated_at": "2026-07-17T00:00:00Z", + "engines": { + "PyJHora": { + "status": "available", + "boundary": "Public parity reports are integrity-verified here; each scoped comparison retains its own replay boundary." + }, + "jyotishganit": { + "status": "available", + "boundary": "Local engine availability is not a claim of external outcome accuracy." + }, + "VedAstro": { + "status": "blocked", + "boundary": "Official external replay closure remains unavailable in this public release; no production tuning is authorized." + }, + "JHora": { + "status": "blocked", + "official_raw_status": "not_collected", + "boundary": "No redistributable JHora desktop raw evidence packet is versioned in this public release." + } + }, + "release_boundaries": { + "external_oracle_closure": false, + "prediction_accuracy_verified": false, + "production_tuning_allowed": false + }, + "assets": [ + { + "id": "oracle_closure_master_dashboard", + "path": "docs/research/oracle_closure_master_dashboard_latest.md", + "sha256": "2c020f0f499abc48a037cd33cc8d2ff2ec3170527793dba9d42792782d7d3868", + "scope": "target-set oracle closure boundary" + }, + { + "id": "public_benchmark_dashboard", + "path": "docs/research/public_benchmark_dashboard_latest.md", + "sha256": "5733b610d763836f3e44d4f7baad7915224322b725014656123db4ebf0c1c3db", + "scope": "public benchmark coverage summary" + }, + { + "id": "pyjhora_same_chart_parity", + "path": "docs/research/pyjhora_same_chart_parity_2026_07_12.md", + "sha256": "c786c5c1fde22d58e029340f0e332a4665587a1000be074a0f625735d3907b84", + "scope": "same-chart PyJHora parity" + }, + { + "id": "pyjhora_varga_ashtakavarga_shadbala_parity", + "path": "docs/research/pyjhora_d2_d4_ashtakavarga_shadbala_parity_2026_07_15.md", + "sha256": "ffdfa08ecb244e7ca751152ca2b18b6952f337e378099c68416b9e9e44a64890", + "scope": "D2/D4, Ashtakavarga, and Shadbala PyJHora parity" + }, + { + "id": "vedastro_parity_matrix", + "path": "docs/research/vedastro_parity_matrix_latest.json", + "sha256": "7be19440e491b62500964fa17c22adb5fc15b0c53af02e14dcbf4940b60ac328", + "scope": "VedAstro closure status matrix" + }, + { + "id": "vedastro_fast_path_checklist", + "path": "docs/research/vedastro_fast_path_checklist_latest.json", + "sha256": "fca4af90a32e990fe8c94804d776b55e57c87a2e26801c34f2528faa1abd10b3", + "scope": "VedAstro official replay prerequisites" + }, + { + "id": "vedastro_parity_matrix_rendered", + "path": "docs/research/vedastro_parity_matrix_latest.md", + "sha256": "a0546572c4f9809167839d675672c5d343101c748d6b9a04fc1c55593fdeadbc", + "scope": "human-readable VedAstro closure status" + }, + { + "id": "vedastro_fast_path_rendered", + "path": "docs/research/vedastro_fast_path_checklist_latest.md", + "sha256": "7f5ff0242278eca7eacd15e005c15930d8bae6896c79c68547c3f0fbb9ba4a58", + "scope": "human-readable VedAstro replay prerequisites" + } + ] +} diff --git a/scripts/external_validation_release_gate.py b/scripts/external_validation_release_gate.py new file mode 100644 index 00000000..11c330e6 --- /dev/null +++ b/scripts/external_validation_release_gate.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 +"""Verify the commercial public external-validation evidence release.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +DEFAULT_MANIFEST = ROOT / "references" / "evidence_manifests" / "commercial_external_validation_release.v1.json" + + +def sha256_file(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def evaluate_manifest(manifest_path: Path = DEFAULT_MANIFEST) -> dict: + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + assets = [] + for item in manifest["assets"]: + path = ROOT / item["path"] + observed = sha256_file(path) if path.is_file() else None + integrity = "verified" if observed == item["sha256"] else ("missing" if observed is None else "mismatch") + assets.append( + { + "id": item["id"], + "path": item["path"], + "scope": item["scope"], + "expected_sha256": item["sha256"], + "observed_sha256": observed, + "integrity": integrity, + } + ) + + verified = sum(item["integrity"] == "verified" for item in assets) + boundaries = manifest["release_boundaries"] + return { + "artifact_id": manifest["artifact_id"], + "release_scope": manifest["release_scope"], + "status": "pass" if verified == len(assets) else "blocked", + "assets": assets, + "engines": manifest["engines"], + "summary": { + "assets_total": len(assets), + "assets_verified": verified, + "external_oracle_closure": boundaries["external_oracle_closure"], + "prediction_accuracy_verified": boundaries["prediction_accuracy_verified"], + "production_tuning_allowed": boundaries["production_tuning_allowed"], + }, + "boundary": "A passing release validates only versioned public evidence integrity. It does not close external oracles, verify prediction accuracy, or authorize production tuning.", + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--manifest", type=Path, default=DEFAULT_MANIFEST) + parser.add_argument("--format", choices=("text", "json"), default="text") + parser.add_argument("--require-match", action="store_true") + args = parser.parse_args() + report = evaluate_manifest(args.manifest) + if args.format == "json": + print(json.dumps(report, ensure_ascii=False, indent=2)) + else: + print(f"external validation release: {report['status']}") + print(f"assets: {report['summary']['assets_verified']}/{report['summary']['assets_total']} verified") + print(f"VedAstro: {report['engines']['VedAstro']['status']}") + print(f"production_tuning_allowed: {report['summary']['production_tuning_allowed']}") + return 0 if report["status"] == "pass" or not args.require_match else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/run_quality_gate.py b/scripts/run_quality_gate.py index 3656dc1a..546a852e 100644 --- a/scripts/run_quality_gate.py +++ b/scripts/run_quality_gate.py @@ -512,6 +512,7 @@ def main() -> int: ROOT / "scripts" / "diagnose_external_engine_adapters.py", ROOT / "scripts" / "interpretation_source_runtime_coverage.py", ROOT / "scripts" / "sync_final_evidence_packet_status.py", + ROOT / "scripts" / "external_validation_release_gate.py", ]: py_compile.compile(str(target), doraise=True) print(f"compiled {target.relative_to(ROOT)}") @@ -519,6 +520,7 @@ def main() -> int: run([PYTHON, "scripts/interpretation_source_inventory_gate.py"]) run([PYTHON, "scripts/diagnose_vedastro_mode.py", "--json"]) run([PYTHON, "scripts/diagnose_external_engine_adapters.py", "--json"]) + run([PYTHON, "scripts/external_validation_release_gate.py", "--require-match"]) else: compile_targets() validate_json_files() diff --git a/tests/test_external_validation_release_gate.py b/tests/test_external_validation_release_gate.py new file mode 100644 index 00000000..b4c25502 --- /dev/null +++ b/tests/test_external_validation_release_gate.py @@ -0,0 +1,53 @@ +"""Regression coverage for the public external-validation release boundary.""" + +from __future__ import annotations + +import json +from pathlib import Path + +import scripts.external_validation_release_gate as gate + + +ROOT = Path(__file__).resolve().parents[1] +MANIFEST = ROOT / "references" / "evidence_manifests" / "commercial_external_validation_release.v1.json" + + +def _manifest() -> dict: + return json.loads(MANIFEST.read_text(encoding="utf-8")) + + +def _copy_manifest_with_one_bad_digest(tmp_path: Path) -> Path: + manifest = _manifest() + manifest["assets"][0]["sha256"] = "0" * 64 + path = tmp_path / "release.json" + path.write_text(json.dumps(manifest), encoding="utf-8") + return path + + +def test_release_manifest_declares_public_assets_and_external_boundaries() -> None: + manifest = _manifest() + assert manifest["schema_version"] == 1 + assert manifest["release_scope"] == "public_research_evidence_snapshot" + assert manifest["engines"]["PyJHora"]["status"] == "available" + assert manifest["engines"]["jyotishganit"]["status"] == "available" + assert manifest["engines"]["VedAstro"]["status"] == "blocked" + assert manifest["engines"]["JHora"]["official_raw_status"] != "verified" + + +def test_evaluate_manifest_accepts_current_public_release() -> None: + report = gate.evaluate_manifest(MANIFEST) + assert report["status"] == "pass" + assert report["summary"]["assets_verified"] == report["summary"]["assets_total"] + assert report["summary"]["production_tuning_allowed"] is False + assert report["engines"]["VedAstro"]["status"] == "blocked" + + +def test_evaluate_manifest_reports_digest_drift(tmp_path: Path) -> None: + report = gate.evaluate_manifest(_copy_manifest_with_one_bad_digest(tmp_path)) + assert report["status"] == "blocked" + assert report["assets"][0]["integrity"] == "mismatch" + + +def test_runtime_truth_profile_runs_external_validation_release_gate() -> None: + text = (ROOT / "scripts" / "run_quality_gate.py").read_text(encoding="utf-8") + assert "external_validation_release_gate.py" in text