fix(ci): migrate staging automatically before deploy
Independent Staging Quality Gate / validate (push) Successful in 10m6s
Independent Staging Quality Gate / publish (push) Successful in 4m18s

Quality gate now dispatches Migrate Staging Database, waits for success, then dispatches Deploy staging. Automatic migrate attests the in-progress gate run so the two jobs cannot deadlock. Manual migrate is unchanged. Staging schema changes must stay backward-compatible with the currently deployed app.
This commit is contained in:
jesse-ux
2026-09-15 23:36:09 +08:00
parent 1a73f64ecd
commit 69ede4367f
8 changed files with 192 additions and 27 deletions
+10 -11
View File
@@ -191,10 +191,10 @@ After source sync and before `up`, the workflow validates `.env.staging` mode/se
1. Complete the server and Gitea bootstrap: create both mode-`0600` env files, preload the reviewed `postgres:17-alpine` image, and configure the listed Actions variables/secrets. No repository-level Supabase variables are required. Deployment and migration workflows use `--pull never` for PostgreSQL, so database image upgrades remain an explicit operator-controlled maintenance action rather than an application-deploy side effect.
2. Push the reviewed test revision directly to `staging`. It may advance independently of `main`; do not merge or reset `main` merely to satisfy staging release mechanics.
3. The `Independent Staging Quality Gate` runs for that push and, when successful, publishes API/web images plus an artifact binding the exact SHA to both immutable image digests and the allowlisted staging controller bundle.
4. The successful publish job rechecks that `staging` still points at the exact SHA, then dispatches `Deploy staging` from `refs/heads/staging` with the source gate run ID. The deploy workflow waits for that gate's final success, validates the artifact, and performs the normal forward-only release under the shared staging host lock.
4. The successful publish job rechecks that `staging` still points at the exact SHA, then dispatches `Migrate Staging Database` for that SHA, waits for it to succeed (no-op when nothing is pending), and only then dispatches `Deploy staging`. You do not click either button for a normal push. Migration failure fails the gate and does not deploy.
5. If environment validation fails, fix the server-side env files without committing or copying secrets, then manually rerun `Deploy staging` using **Use workflow from: staging**, the same successful SHA in `deploy_sha`, an empty `gate_run_id`, and `allow_rollback=false`; the workflow resolves a successful exact-SHA staging push gate before mutation.
6. If the read-only checker reports a pending migration, stop app deployment and run `Migrate Staging Database` manually using **Use workflow from: staging**. Leave `deploy_sha` empty unless you are rolling back to an earlier gated SHA. Migration success does not dispatch deployment.
7. After migration succeeds, manually start `Deploy staging` from `staging` with that same exact SHA and `allow_rollback=false`, then confirm `https://staging.jyotisha.chat/api/health` reports it and private API health.
6. The **Migrate Staging Database** button remains for rollback catch-up and one-off data repairs. Leave `deploy_sha` empty unless you are targeting an earlier gated SHA. A manual run does not dispatch deployment.
7. Confirm `https://staging.jyotisha.chat/api/health` reports the SHA from step 2.
### Resetting one staging account
@@ -261,16 +261,15 @@ PostgreSQL is private: `deploy/docker-compose.postgres.yml` has no `ports` mappi
Use this order for every staging revision:
1. Review the test change, then push its exact commit directly to `staging`; `main` may remain at a different SHA.
2. Wait for `Independent Staging Quality Gate` to pass and publish that exact full SHA's API/web digest and controller artifact. Its publish job dispatches the staging-ref deployment and refuses dispatch if `staging` already advanced.
3. The dispatched `Deploy staging` workflow validates the source gate run and checks the exact SHA in read-only migration-check mode before changing API, web, or Caddy. If it reports pending or drifted migrations, stop; do not retry the application deployment as if it were a migration.
4. Open **Migrate Staging Database -> Run workflow**, select **Use workflow from: staging**, and leave `deploy_sha` empty. The workflow resolves the latest staging commit that has a successful `backend-quality-gate` push run, then applies the same exact-SHA gate check as a filled SHA. Fill the 40-character SHA only when rolling back to an earlier gated revision. If `staging` is already ahead, the gate-attested `deploy/is-docs-only-range.sh` must prove the extra commits are docs-only; a gated-path advance still refuses. It starts only PostgreSQL and runs the digest-pinned migrator from the gate-attested controller bundle. Check the first log line `resolved deploy_sha=…` to see which revision will migrate.
5. A successful migration rechecks that `staging` is still the migrated SHA or only docs-only ahead, then prints the ordered migration ledger, but does not dispatch deployment. The operator must then open **Deploy staging -> Run workflow**, select **Use workflow from: staging**, enter the same exact SHA in `deploy_sha` (the value printed as `resolved deploy_sha`), leave `gate_run_id` empty, and set `allow_rollback=false`. If `staging` advanced by a gated path, stop rather than substituting a branch name, short SHA, or newer commit.
2. Wait for `Independent Staging Quality Gate` to pass and publish that exact full SHA's API/web digest and controller artifact. Its publish job first dispatches `Migrate Staging Database` for that SHA and waits for success (a no-op when nothing is pending), then dispatches `Deploy staging`. Do not click either button for a normal push. If `staging` already advanced by a gated path, dispatch is refused.
3. The dispatched `Deploy staging` workflow still validates the source gate run and checks the exact SHA in read-only migration-check mode before changing API, web, or Caddy. After the automatic migrate, that check should be clean. If it reports pending or drifted migrations, stop; do not retry the application deployment as if it were a migration.
4. Use **Migrate Staging Database** manually only for rollback catch-up or a one-off data repair. Leave `deploy_sha` empty unless targeting an earlier gated SHA. A manual run does not dispatch deployment.
5. Confirm `https://staging.jyotisha.chat/api/health` and verify that its deployment SHA is the SHA from step 2.
6. After health verification, create the local encrypted backup described below.
**Deploy Production** and **Migrate Production Database** still require a hand-filled 40-character SHA, plus `allow_rollback` / recovery attestation. That is a guardrail: production must name the exact revision out loud. Do not copy the staging empty-SHA shortcut onto those two buttons.
6. Confirm `https://staging.jyotisha.chat/api/health` and verify that its deployment SHA is the SHA from step 2.
7. After health verification, create the local encrypted backup described below.
**Deploy Production** and **Migrate Production Database** stay fully manual. They still require a hand-filled 40-character SHA plus `allow_rollback` / recovery attestation (`restore_verified` is a person's guarantee that the recovery point restores). Do not copy the staging automatic migrate-then-deploy shortcut onto production.
The deploy and migration workflows share the `staging-mutation` Actions concurrency group, and their live-tree sync plus Compose work runs under `/opt/jyotisha-staging/.state/mutation.lock`. The synchronized tree explicitly preserves `/backups/`, `.env*`, `.state`, and `.incoming`. The read-only checker exits before app changes when a migration is pending. Its message includes the exact SHA and the `Migrate Staging Database` workflow name. A failed migration does not re-dispatch deployment. Application rollback restores the previously recorded digest references and SHA, falling back to validated local image IDs only when transitioning from the pre-foundation local-image deployment; it does not roll back database state.
The deploy and migration workflows share the `staging-mutation` Actions concurrency group, and their live-tree sync plus Compose work runs under `/opt/jyotisha-staging/.state/mutation.lock`. The synchronized tree explicitly preserves `/backups/`, `.env*`, `.state`, and `.incoming`. The read-only checker still exits before app changes when a migration is pending. After the automatic pre-deploy migrate that should be rare. A failed automatic migration fails the quality gate and does not dispatch deployment. Application rollback restores the previously recorded digest references and SHA, falling back to validated local image IDs only when transitioning from the pre-foundation local-image deployment; it does not roll back database state.
### Production recovery point before schema migration
+13
View File
@@ -89,6 +89,19 @@ compose=("${docker_command[@]}" compose -p jyotisha-staging -f deploy/docker-com
"${compose[@]}" up -d --no-build --pull never --wait postgres
"${compose[@]}" exec -T postgres psql -v ON_ERROR_STOP=1 -U postgres -d jyotisha \
-f /dev/stdin < deploy/postgres/002-ensure-business-compatibility-roles.sql
set +e
pending_output="$("${compose[@]}" --profile migration-check run --rm migration-checker 2>&1)"
pending_status=$?
set -e
if [ "$pending_status" -eq 0 ]; then
echo "无待应用迁移"
elif [ "$pending_status" -eq 3 ]; then
echo "待应用迁移:"
printf '%s\n' "$pending_output"
else
echo "待应用迁移检查退出码 $pending_status;继续交给 migrator"
printf '%s\n' "$pending_output"
fi
"${compose[@]}" --profile migration run --rm migrator
"${compose[@]}" exec -T postgres psql -U postgres -d jyotisha -Atc \
'select filename from migration.schema_migrations order by filename'