fix: repair production recovery lock ownership

deploy/README.md

@@ -254,7 +254,7 @@ The deploy and migration workflows share the `staging-mutation` Actions concurre
 
 Use Gitea Actions → **Create Production Recovery Point** from the exact current `main` release SHA before every production schema migration. The manual workflow requires `main`, `staging`, public staging health, and the successful release gate to identify the same SHA. It uses the pinned production SSH identity, shares the `production-mutation` lock, and runs `deploy/run-production-recovery.sh` with shell tracing disabled.
 
-The host script writes an AES-256-CBC/PBKDF2 encrypted custom-format PostgreSQL dump under `/opt/jyotisha-production/backups`, restores it into a uniquely named disposable database, validates non-sensitive row/table counts, removes only that disposable database, and writes a mode-`0600` verification manifest. The workflow retrieves only the encrypted dump and verification metadata, verifies the SHA-256 digest, and uploads them as a 30-day Gitea Actions artifact with compression disabled. The artifact-backed `recovery_reference`, `recovery_created_at`, and `restore_verified=true` output are the inputs for **Migrate Production Database**. Do not use the attestation if backup, restore, retrieval, digest validation, or artifact upload fails.
+The host script writes an AES-256-CBC/PBKDF2 encrypted custom-format PostgreSQL dump under `/opt/jyotisha-production/backups`, restores it into a uniquely named disposable database, validates non-sensitive row/table counts, removes only that disposable database, and writes a mode-`0600` verification manifest. Before acquiring the shared host lock it validates that `.state` and `backups` are real directories, then uses the constrained deployment-ownership sudo boundary to restore only those directories and an existing regular `mutation.lock` to the current `deploy` UID/GID; it never removes or replaces a lock inode. The workflow retrieves only the encrypted dump and verification metadata, verifies the SHA-256 digest, and uploads them as a 30-day Gitea Actions artifact with compression disabled. The artifact-backed `recovery_reference`, `recovery_created_at`, and `restore_verified=true` output are the inputs for **Migrate Production Database**. Do not use the attestation if backup, restore, retrieval, digest validation, ownership normalization, or artifact upload fails.
 
 Never restore over `jyotisha`, delete the PostgreSQL volume, print `.env.production.database`, expose `PRODUCTION_BACKUP_ENCRYPTION_KEY`, or substitute a staging recovery artifact. The encrypted local archive is preserved for repair; the Actions artifact supplies the required off-host copy.
 

deploy/run-production-recovery.sh

@@ -18,18 +18,35 @@ umask 077
 state_directory="$DEPLOY_PATH/.state"
 backup_directory="$DEPLOY_PATH/backups"
 environment_file="$DEPLOY_PATH/.env.production.database"
+lock_file="$state_directory/mutation.lock"
 
 [ -f "$environment_file" ] && [ ! -L "$environment_file" ] || {
   echo "production database environment file is missing or unsafe" >&2
   exit 1
 }
+for directory in "$state_directory" "$backup_directory"; do
+  if [ -e "$directory" ]; then
+    [ -d "$directory" ] && [ ! -L "$directory" ] || {
+      echo "production state or backup directory is unsafe" >&2
+      exit 1
+    }
+  fi
+done
 install -d -m 700 "$state_directory" "$backup_directory"
-[ ! -L "$state_directory" ] && [ ! -L "$backup_directory" ] || {
-  echo "production state or backup directory is unsafe" >&2
-  exit 1
-}
+deployment_uid="$(id -u)"
+deployment_gid="$(id -g)"
+sudo -n chown "$deployment_uid:$deployment_gid" "$state_directory" "$backup_directory"
+chmod 700 "$state_directory" "$backup_directory"
+if [ -e "$lock_file" ]; then
+  [ -f "$lock_file" ] && [ ! -L "$lock_file" ] || {
+    echo "production mutation lock is unsafe" >&2
+    exit 1
+  }
+  sudo -n chown "$deployment_uid:$deployment_gid" "$lock_file"
+  chmod 600 "$lock_file"
+fi
 
-exec 9>"$state_directory/mutation.lock"
+exec 9>"$lock_file"
 flock -n 9 || {
   echo "another production mutation holds the host lock" >&2
   exit 75

docs/BUG_HISTORY.md

@@ -3408,3 +3408,17 @@
 - 防复发：出生时段必须继续按 `period_only` 诚实保存，不能转换成时段中点、`00:00` 或任何候选分钟。无分钟“每日运势”只能使用服务器公共 Panchanga，必须明确它不是个人命盘日运；不得声称个人上升点、宫位、分盘、大运、本命过境叠加、确定事件或精确时间。
 - 相关记录：BUG-127、BUG-198、BUG-200
 - 修复版本：本次 staging 修复提交（精确 SHA 以远端分支与 staging 质量门禁结果为准）
+
+## BUG-203 | 生产恢复点因 `.state/mutation.lock` 所有权漂移无法创建
+
+- 状态：resolved（待重新通过 staging、release gate 与生产恢复验收）
+- 首次发现：2026-08-15
+- 最近更新：2026-08-15
+- 影响面：`Create Production Recovery Point`、生产 schema migration 前置恢复门禁，以及后续 production deploy。
+- 用户现象：`main` 与 `staging` 已同步且 release gate 成功，但生产恢复 workflow Run 1865 在创建备份前失败，日志为 `/opt/jyotisha-production/.state/mutation.lock: Permission denied`；迁移和部署因此保持阻断，生产运行版本未改变。
+- 根因：生产 bootstrap 遗留的 `.state` 路径或既有 lock 仍为非 `deploy` 所有。原恢复脚本只执行 `install -d -m 700`；对已经存在的目录该命令不会恢复所有权，随后由 `deploy` 打开共享锁即被内核拒绝。失败发生在 `pg_dump` 前，因此本轮没有生成可用恢复证明，也没有执行 schema migration。
+- 修复：恢复脚本先对 `.state`、`backups` 和既有 `mutation.lock` 做类型与非符号链接校验，再通过已审查的部署树 sudo 边界仅恢复这些路径到当前 `deploy` UID/GID 和私有权限；不递归改动备份文件、不删除或替换 lock inode，随后仍用同一个 `flock -n` fail-closed 获取共享 host lock。同步更新生产 runbook 和静态安全契约测试。
+- 验证：本地 `bash -n`、YAML parse、聚焦 workflow contract 与 `git diff --check` 必须通过；远端必须重新完成 staging quality/deploy、exact-SHA release gate、真实 production dump + disposable restore + off-site artifact，再允许 migration/deploy。
+- 防复发：生产私有状态目录必须保持 `deploy:deploy 0700`，共享 lock 必须是普通非 symlink 文件且不可通过删除重建来“修复”；任何恢复流程失败都不得手填 `restore_verified=true` 或跳过恢复门禁。
+- 相关记录：生产迁移 runbook、Run 1865
+- 修复版本：待提交（精确 SHA 以重新发布后的远端分支与 production health 为准）

docs/operations/production-server-migration-2026-08.md

@@ -90,7 +90,7 @@ Use distinct production credentials for PostgreSQL roles, Better Auth, Resend, b
 
 ## Database migration engineering gate
 
-Before importing data, dispatch Gitea Actions → `Create Production Recovery Point` for the exact accepted release SHA. It creates an AES-256-CBC/PBKDF2 encrypted custom-format dump on the production host, restores it into a disposable database, verifies non-sensitive table counts, removes only the disposable database, and uploads the encrypted dump plus verification manifest as a 30-day Gitea Actions artifact. The workflow prints the non-sensitive `recovery_reference` and exact UTC `recovery_created_at`; use those values only after the artifact upload succeeds.
+Before importing data, dispatch Gitea Actions → `Create Production Recovery Point` for the exact accepted release SHA. It validates the private production `.state` and `backups` paths, restores only their documented `deploy` ownership and an existing regular shared lock when bootstrap ownership has drifted, creates an AES-256-CBC/PBKDF2 encrypted custom-format dump on the production host, restores it into a disposable database, verifies non-sensitive table counts, removes only the disposable database, and uploads the encrypted dump plus verification manifest as a 30-day Gitea Actions artifact. It must preserve the existing lock inode and fail closed on symlinks or non-regular lock paths. The workflow prints the non-sensitive `recovery_reference` and exact UTC `recovery_created_at`; use those values only after the artifact upload succeeds.
 
 Then dispatch Gitea Actions → `Migrate Production Database` for the same exact accepted release SHA. The workflow requires `main == staging == deploy_sha`, the same successful staging backend gate, the same manual release gate, and the public staging `/api/health` identity for that SHA. It also requires the recovery workflow's non-sensitive reference, exact UTC creation time, and `restore_verified=true`; the recovery point must be no more than 24 hours old and must already have passed the restore verification. It verifies the current production revision, obtains the gate-attested immutable Web image, runs the schema checker, applies only pending application schema migrations, and requires the checker to converge afterward.
 

frontend/tests/staging-backend-workflows.test.ts

@@ -1085,6 +1085,11 @@ test("production recovery workflow creates a verified encrypted off-site artifac
   assert.doesNotMatch(workflow, /STAGING_BACKUP_ENCRYPTION_KEY|\.env\.production\.database[^\n]*(?:cat|awk)/);
 
   assert.match(runner, /^#!\/usr\/bin\/env bash\nset -euo pipefail\nset \+x\n/);
+  assert.match(runner, /for directory in "\$state_directory" "\$backup_directory"/);
+  assert.match(runner, /sudo -n chown "\$deployment_uid:\$deployment_gid" "\$state_directory" "\$backup_directory"/);
+  assert.match(runner, /production mutation lock is unsafe/);
+  assert.match(runner, /sudo -n chown "\$deployment_uid:\$deployment_gid" "\$lock_file"/);
+  assert.doesNotMatch(runner, /rm -f[^\n]*mutation\.lock|chown -R/);
   assert.match(runner, /another production mutation holds the host lock/);
   assert.match(runner, /usage_percent < 70/);
   assert.match(runner, /pg_dump[\s\S]*--format=custom --no-owner --no-acl/);