docs(staging): close bounded fetch incident

Record the tested staging evidence and explicit browser handoff blockers.

.gitea/workflows/deploy-staging.yml

@@ -96,7 +96,19 @@ jobs:
           git init .
           git remote remove origin 2>/dev/null || true
           git remote add origin https://git.copse.top/root/Jyotisha.git
-          git fetch --no-tags origin main "$DEPLOY_SHA"
+          fetch_succeeded=false
+          for attempt in 1 2 3; do
+            if timeout 120 git fetch --no-tags origin main "$DEPLOY_SHA"; then
+              fetch_succeeded=true
+              break
+            fi
+            if [ "$attempt" -eq 3 ]; then
+              echo "trusted main fetch failed after $attempt bounded attempts" >&2
+              exit 1
+            fi
+            sleep $((attempt * 10))
+          done
+          [[ "$fetch_succeeded" == true ]]
           git checkout --detach --force origin/main
           git merge-base --is-ancestor "$DEPLOY_SHA" HEAD || { echo "staging revision is not in trusted main history" >&2; exit 1; }
 

.gitea/workflows/migrate-staging-database.yml

@@ -68,7 +68,19 @@ jobs:
           git init .
           git remote remove origin 2>/dev/null || true
           git remote add origin https://git.copse.top/root/Jyotisha.git
-          git fetch --no-tags origin main "$DEPLOY_SHA"
+          fetch_succeeded=false
+          for attempt in 1 2 3; do
+            if timeout 120 git fetch --no-tags origin main "$DEPLOY_SHA"; then
+              fetch_succeeded=true
+              break
+            fi
+            if [ "$attempt" -eq 3 ]; then
+              echo "trusted main fetch failed after $attempt bounded attempts" >&2
+              exit 1
+            fi
+            sleep $((attempt * 10))
+          done
+          [[ "$fetch_succeeded" == true ]]
           git checkout --detach --force origin/main
           git merge-base --is-ancestor "$DEPLOY_SHA" HEAD || { echo "staging revision is not in trusted main history" >&2; exit 1; }
 

docs/BUG_HISTORY.md

@@ -2223,4 +2223,20 @@
 - 防复发：禁止 staging workflow 直接注入多行私钥或打印 decoded secret 变量；env owner 必须由部署树身份决定，root 受控脚本不得用 root 临时文件改变持久 env owner。任何凭据日志暴露先轮换/撤销/清理，再修代码和重跑。
 - 相关记录：BUG-124、BUG-127、ERR-092、ERR-093、ERR-094
 - 复发自：无
-- 修复版本：待安全修复 PR 与 staging 验收
+- 修复版本：`f7a615a5bf11ed95b3a6c7e6d28dfe8150a825ef`；staging migration/deploy 与安全验收完成
+
+## BUG-129 | staging trusted-main checkout 无界 fetch 导致自动部署长期占用 mutation queue
+
+- 状态：resolved
+- 首次发现：2026-08-06
+- 最近更新：2026-08-06
+- 影响面：Gitea staging deploy/migration 控制器的 trusted-main checkout；production 与 staging 应用数据面未受影响。
+- 用户现象：exact-SHA quality gate run `1473` 成功后，自动 deploy run `1474` 在 `git fetch --no-tags origin main "$DEPLOY_SHA"` 长时间没有日志进展；期间 staging 公网与 `.state/deployed-revision` 均保持上一健康 SHA，SSH/远端 mutation 尚未开始。fetch 后续自行恢复，run 最终于 18 分钟完成并成功部署 `02cc483b7c303e6cc0f26fb31462c50adb007f12`，5 个容器 restart count 均为 0。
+- 触发条件：Gitea runner 在空仓库中抓取 reviewed `main` 和目标 SHA 时传输长时间停滞；底层远端/网络原因未闭环。checkout step 没有命令级 timeout/retry，只依赖 30 分钟 job 总超时。
+- 根因：已确认的控制面根因是发布控制器对镜像拉取有 bounded retry，却对 trusted-main Git fetch 采用无界单次调用；无论底层传输为何停滞，该调用都会长期占用 `staging-mutation` queue，并推迟后续受控重试。底层 Git 传输停滞原因保持 blocked，不作推断。
+- 修复：Gitea deploy 与 migration 的 trusted-main fetch 改为最多 3 次、每次 120 秒的 bounded retry，重试间隔递增；耗尽后明确 fail closed。仍只 checkout `origin/main`，仍要求目标 SHA 是 reviewed main ancestor，不改变 exact-SHA artifact、forward-only 或 manual rollback 边界。
+- 验证：原 run `1474` 最终成功部署 `02cc483b7c303e6cc0f26fb31462c50adb007f12`，公网 health、主机 state 与 `main/staging` 精确一致，5 个容器 restart count 均为 0；本地 staging workflow contracts 31/31、修改后 YAML 解析、mandatory pre-work 和 diff 检查通过；Gitea PR quality gate run `1475` 成功。修复后 exact-SHA staging 控制面部署仍按正常发布流程单独核对，不将原 run 的自行恢复冒充为 bounded retry 已在真实故障中触发。
+- 防复发：所有 release-controller 网络调用必须同时具备命令级上限和失败闭合；不得仅依赖 job 总 timeout。回归测试必须同时覆盖 deploy 和 migration 的 attempt 数、单次 timeout、最终错误和 ancestry check。
+- 相关记录：BUG-128、ERR-094、ERR-095
+- 复发自：无
+- 修复版本：`52b467cbe56f820a6acc19bb3fe3ebbf4ad55776`；待合并后由 staging exact-SHA 发布采用

docs/operations/personal-report-staging-acceptance-2026-08-06.md

@@ -0,0 +1,140 @@
+# Personal Report Staging Acceptance — 2026-08-06
+
+This record contains only release identities, aggregate operational evidence, schema metadata, stable status codes, and synthetic transaction counts. It contains no user identity, birth data, report body, OTP, password, cookie, JWT, model key, database credential, SSH private key, or raw environment file.
+
+## Release identity
+
+- Pre-change application baseline: `49da8f916960030d5760d8dedf4e77820732a527`
+- Read-only upstream Skill source commit: `unknown` (source directory has no usable Git metadata)
+- Read-only upstream tree SHA-256: `9034e1967032d09c7fbae83fc2205f7e75e8ad482c5f9eba1bf309fe30aef5bb`
+- Personal-report implementation merge: `e018dc90a73d49596563b5ae2b5fc203402cc86a`
+- Security/control-plane merge: `f7a615a5bf11ed95b3a6c7e6d28dfe8150a825ef`
+- Staging migration and application-under-test SHA: `f7a615a5bf11ed95b3a6c7e6d28dfe8150a825ef`
+- Docs-only acceptance attestation deployment SHA: `02cc483b7c303e6cc0f26fb31462c50adb007f12`
+- Application rollback target: `49da8f916960030d5760d8dedf4e77820732a527`, subject to retained successful gate artifacts. The additive database migration remains in place after application rollback.
+
+At deployment verification, Gitea `main`, Gitea `staging`, `/opt/jyotisha-staging/.state/deployed-revision`, and public `/api/health` all reported the same full SHA.
+
+## Automated quality evidence
+
+### Local
+
+- Mandatory `scripts/pre_work_check.py`: pass, including fragment scan, external-engine adapter diagnostic, remote visibility, and focused governance tests.
+- Personal-report frontend suite: 142/142.
+- Core Python report/import/orchestrator matrix: 87/87.
+- TypeScript: pass.
+- Next production build: pass; NFT whole-project warning count 0.
+- ESLint: 0 errors; 4 unrelated pre-existing warnings.
+- Staging workflow/security contracts after credential hardening: 31/31.
+- JSON Schema/Zod/Python contract, dual migration, owner service, API, entitlement, grounded generation, reader, D1 SVG, and browser print contracts: pass.
+
+The complete frontend suite could not be made fully executable on the local macOS host because Docker CLI and the bare `python` command were absent. This was not reported as green locally.
+
+### Gitea complete runner
+
+- PR quality gate `1459`: success; Python quick gate 291 passed / 1 skipped and frontend 1409/1409, including real Docker/PostgreSQL migration fixture.
+- Final personal-report PR quality gate `1461`: success.
+- Security-control-plane PR quality gates `1465` and `1467`: success, including Docker/PostgreSQL fixtures.
+- Final `staging` push quality gate `1469`: success for full SHA `f7a615a5bf11ed95b3a6c7e6d28dfe8150a825ef`; immutable API/web manifest published.
+- Automatic deploy check `1470`: stopped safely with exit 3 because the report migration was pending. Its logs showed the base64 SSH secret as masked and no private-key header/material.
+- Manual migration `1471`: success; `20260806000000_personal_reports.sql` applied and present once in the migration ledger.
+- Manual deploy `1472`: success with the same exact application-under-test SHA and `allow_rollback=false`.
+- Docs-only attestation gate `1473`: success for `02cc483b7c303e6cc0f26fb31462c50adb007f12`; immutable manifest published.
+- Docs-only attestation deploy `1474`: success. The trusted-main fetch paused for an extended period before recovering; no SSH/staging mutation occurred during the pause. Public health and host state then moved to the exact attestation SHA with zero container restarts. See `BUG-129` / `ERR-095` for the bounded-fetch follow-up.
+
+## Security incident and containment
+
+Before the final release, an earlier failed staging-only deploy exposed the then-current multiline staging SSH key in Gitea logs. The application had not switched and production was not involved.
+
+Containment completed before any rerun:
+
+- generated and verified a new staging ED25519 key;
+- revoked the exposed authorized key and proved it no longer authenticated;
+- deleted the old local key;
+- replaced the Gitea and GitHub staging secrets;
+- deleted 28 potentially affected Gitea deploy/migration runs;
+- retained the quality-gate run and immutable image artifact, which never received the SSH secret;
+- changed the staging secret contract to one-line base64, decoded only into a mode-0600 temporary key and validated by `ssh-keygen`;
+- fixed `.env.staging*` ownership validation to use the deployment-tree owner and fixed rollout replacement to preserve owner/gid.
+
+The staging host key had also changed before inspection. Strict SSH was paused until the observed ED25519 key exactly matched the independently administered Gitea `STAGING_KNOWN_HOSTS` value. Strict checking was never disabled. See `ERR-092`, `ERR-093`, `ERR-094`, and `BUG-128`.
+
+## Migration and authorization evidence
+
+Post-deploy schema inspection reported:
+
+- `public.personal_reports` exists;
+- RLS enabled;
+- policies: owner SELECT and owner DELETE;
+- `authenticated`: SELECT, DELETE only;
+- `service_role`: SELECT, INSERT, UPDATE, DELETE;
+- personal-report migration ledger count: 1.
+
+A two-owner synthetic RLS test ran entirely inside one PostgreSQL transaction and then rolled back:
+
+- owner SELECT count: 1;
+- cross-owner SELECT count: 0;
+- cross-owner DELETE count: 0;
+- owner DELETE count: 1;
+- persisted synthetic users after rollback: 0;
+- persisted synthetic reports after rollback: 0.
+
+No report document or birth fact was needed or persisted for this check.
+
+## Health and unauthenticated smoke
+
+- Public `/api/health`: `ok`; deployment SHA exact match.
+- `/login`: 200.
+- Logged-out `/api/account`: 401.
+- Logged-out report GET and POST: 401.
+- Logged-out report reader route: reachable; its data API remains authenticated.
+- Internal Python `/api/health`: 200, `status=ok`, `swisseph_available=true`.
+- API, web, PostgreSQL, worker, and Caddy containers running; API/web/PostgreSQL health checks healthy.
+- API/web/Caddy recent fatal/report-guard error-signature count: 0.
+- All five staging containers restart count: 0.
+- Server web container: no Chromium, Chrome, Playwright, or Puppeteer executable/process.
+- Staging report selectors: one canonical enabled selector and one canonical daily-limit selector; env files remain `deploy:deploy` mode 0600.
+- Legacy-compatible model API key, HTTPS base URL, model ID, Python API URL, report feature flag, and report limit are present in the web runtime. Values were not printed.
+
+## Idle resource baseline
+
+One post-deploy no-load sample (not a concurrency claim):
+
+| Service | CPU | Memory |
+| --- | ---: | ---: |
+| API | 0.02% | 48.54 MiB |
+| Caddy | 0.35% | 36.15 MiB |
+| PostgreSQL | 0.16% | 44.67 MiB |
+| Rectification worker | 0.30% | 194.8 MiB |
+| Web | 1.69% | 130.3 MiB |
+
+This is an idle snapshot only. It does not satisfy the planned single-user/two-user report-generation performance measurement.
+
+## Backups
+
+- Pre-migration encrypted staging backup: success.
+- Post-deploy encrypted staging backup: success.
+- Retention after post-deploy backup: 3 archives.
+- Backup directory mode: 0700.
+- Latest archive mode: 0600; non-empty (717,008 bytes).
+- No secret was passed in argv or printed.
+
+## Blocked / user handoff
+
+The following are intentionally **not** marked complete because no authorized synthetic browser account credentials were configured in Gitea/GitHub, and existing real users were not borrowed:
+
+1. End-to-end authenticated model report generation from an accepted/confirmed synthetic profile.
+2. Browser verification of summary → themes → evidence order using a real ready document.
+3. Browser cross-owner report URL check (database RLS isolation was verified transactionally).
+4. Desktop Chrome Print → Save as PDF.
+5. macOS Safari Print → Save as PDF.
+6. iPhone Safari Share/Print and WeChat guidance.
+7. 20–40-page pagination, Chinese font, SVG sharpness, and table clipping checks.
+8. Serialized ready-document byte size and generated PDF byte size from a real report.
+9. Single-user generation CPU/memory/duration and two-user concurrent-generation resource evidence.
+
+These require the user/browser handoff in `docs/operations/personal-report-staging.md`. Failure, unavailable model, blocked evidence, or generation timeouts must be recorded as observed; they must not be converted into a success claim.
+
+## Production boundary
+
+No production deployment, production migration, production secret rotation, production database operation, domain change, or Supabase production change was performed.

docs/research/pre_work_error_ledger.md

@@ -145,6 +145,12 @@ A failed exact-SHA staging deploy displayed the multiline staging SSH private ke
 
 Prevention: store `STAGING_SSH_PRIVATE_KEY` only as one unwrapped base64 line; workflows decode it into a mode-`0600` temporary key, validate it with `ssh-keygen`, and delete the temporary directory on every exit. Contract tests must reject direct multiline `SSH_PRIVATE_KEY` injection or `printf` of a decoded secret variable. A leaked staging key must be rotated and revoked before any rerun; production keys remain a separate boundary and were not involved in this incident.
 
+## ERR-095 | Gitea trusted-main fetch can stall until the whole deploy job times out | mitigated 2026-08-06
+
+After exact-SHA staging gate `1473` succeeded, automatic deploy `1474` stopped making log progress for an extended period in the empty-repository `git fetch --no-tags origin main "$DEPLOY_SHA"` step before any SSH or staging mutation. Public and state SHAs remained on the previous healthy release during the stall. The fetch later recovered and the 18-minute run successfully deployed the exact SHA, but the checkout had no command-level bound; without recovery, the 30-minute job timeout was the only escape and the shared mutation queue would remain occupied. The underlying Git transport stall is not proven.
+
+Prevention: wrap the Gitea deploy and migration trusted-main fetch in three 120-second bounded attempts with incremental delay and explicit fail-closed exhaustion. Preserve the reviewed-main ancestry check, exact-SHA gate artifact, forward-only deploy policy, and shared mutation queue. Never treat an in-progress or timed-out fetch as a deployment result.
+
 ## Fragment Sweep Command Set
 
 ## ERR-086 | Steve Jobs jyotishganit artifacts used non-San-Francisco coordinates | mitigated 2026-07-21

frontend/tests/staging-backend-workflows.test.ts

@@ -488,7 +488,10 @@ test("Gitea deploy and migration consume the exact successful gate artifact", ()
     assert.match(workflow, /actions\/artifacts\/\$artifact_id\/zip/);
     assert.match(workflow, /node frontend\/scripts\/staging-image-manifest\.mjs/);
     assert.match(workflow, /name: Checkout trusted main controller/);
-    assert.match(workflow, /git fetch --no-tags origin main "\$DEPLOY_SHA"/);
+    assert.match(workflow, /for attempt in 1 2 3/);
+    assert.match(workflow, /timeout 120 git fetch --no-tags origin main "\$DEPLOY_SHA"/);
+    assert.match(workflow, /trusted main fetch failed after \$attempt bounded attempts/);
+    assert.match(workflow, /\[\[ "\$fetch_succeeded" == true \]\]/);
     assert.match(workflow, /git checkout --detach --force origin\/main/);
     assert.match(workflow, /git merge-base --is-ancestor "\$DEPLOY_SHA" HEAD/);
     assert.doesNotMatch(workflow, /docker manifest inspect/);