root/Jyotisha
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

fix: reuse docker boundary for production recovery
Independent Staging Quality Gate / validate (push) Successful in 13m32s
Details
Independent Staging Quality Gate / publish (push) Successful in 2m13s
Details

Browse Source
This commit is contained in:
Jesse_Chen
2026-08-16 02:42:18 +08:00
parent 2b1deff0e9
commit 29a7295667
5 changed files with 40 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/production-server-migration-2026-08.md
+2 -2
View File
@@ -60,7 +60,7 @@ Do not put the Ubuntu password, database URLs, Resend key, payment key, model-pr
Perform this interactively before any workflow dispatch:
1. Patch Ubuntu and install Docker Engine, Compose v2, `rsync`, `curl`, `jq`, `flock`, and UFW.
2. Create a non-root `deploy` user, install a dedicated Ed25519 public key, and grant only the reviewed passwordless commands needed for Docker and deployment-tree ownership.
2. Create a non-root `deploy` user, install a dedicated Ed25519 public key, and grant only the reviewed passwordless Docker command used by the deployment workflows. Ownership normalization must reuse the capability-constrained helper container described below rather than adding a separate passwordless host `chown` rule.
3. Verify a second key-only session and rotate the exposed bootstrap password. For this host, the production owner explicitly requires password authentication to remain enabled for other operators; do not change `PasswordAuthentication`. Workflows must still use the dedicated deploy key.
4. Permit only the confirmed SSH port plus `80/tcp`, `443/tcp`, and `443/udp`. Do not publish `3000`, `5200`, `5432`, or the Docker API.
5. Create a 2–4 GB swap file and enable Docker log rotation. Keep at least 15 GB free before the first image pull and database import.
@@ -90,7 +90,7 @@ Use distinct production credentials for PostgreSQL roles, Better Auth, Resend, b
## Database migration engineering gate
Before importing data, dispatch Gitea Actions → `Create Production Recovery Point` for the exact accepted release SHA. It validates the private production `.state` and `backups` paths, restores only their documented `deploy` ownership and an existing regular shared lock when bootstrap ownership has drifted, creates an AES-256-CBC/PBKDF2 encrypted custom-format dump on the production host, restores it into a disposable database, verifies non-sensitive table counts, removes only the disposable database, and uploads the encrypted dump plus verification manifest as a 30-day Gitea Actions artifact. It must preserve the existing lock inode and fail closed on symlinks or non-regular lock paths. The workflow prints the non-sensitive `recovery_reference` and exact UTC `recovery_created_at`; use those values only after the artifact upload succeeds.
Before importing data, dispatch Gitea Actions → `Create Production Recovery Point` for the exact accepted release SHA. It validates the private production `.state` and `backups` paths, obtains the immutable image ID of the already-running PostgreSQL container, and reuses the existing passwordless Docker boundary to run that local image with `--pull never`, no network, a read-only root filesystem, `no-new-privileges`, and only `CHOWN` retained. With only `.state` and `backups` bind-mounted, it restores the two directory mount points and an existing regular shared lock to documented `deploy` ownership when bootstrap ownership has drifted. It then creates an AES-256-CBC/PBKDF2 encrypted custom-format dump on the production host, restores it into a disposable database, verifies non-sensitive table counts, removes only that disposable database, and uploads the encrypted dump plus verification manifest as a 30-day Gitea Actions artifact. It must preserve the existing lock inode, avoid recursively changing historical backups, and fail closed on symlinks, non-regular lock paths, or an unexpected image identity. The workflow prints the non-sensitive `recovery_reference` and exact UTC `recovery_created_at`; use those values only after the artifact upload succeeds.
Then dispatch Gitea Actions → `Migrate Production Database` for the same exact accepted release SHA. The workflow requires `main == staging == deploy_sha`, the same successful staging backend gate, the same manual release gate, and the public staging `/api/health` identity for that SHA. It also requires the recovery workflow's non-sensitive reference, exact UTC creation time, and `restore_verified=true`; the recovery point must be no more than 24 hours old and must already have passed the restore verification. It verifies the current production revision, obtains the gate-attested immutable Web image, runs the schema checker, applies only pending application schema migrations, and requires the checker to converge afterward.