fix: reuse docker boundary for production recovery
This commit is contained in:
+1
-1
@@ -254,7 +254,7 @@ The deploy and migration workflows share the `staging-mutation` Actions concurre
|
||||
|
||||
Use Gitea Actions → **Create Production Recovery Point** from the exact current `main` release SHA before every production schema migration. The manual workflow requires `main`, `staging`, public staging health, and the successful release gate to identify the same SHA. It uses the pinned production SSH identity, shares the `production-mutation` lock, and runs `deploy/run-production-recovery.sh` with shell tracing disabled.
|
||||
|
||||
The host script writes an AES-256-CBC/PBKDF2 encrypted custom-format PostgreSQL dump under `/opt/jyotisha-production/backups`, restores it into a uniquely named disposable database, validates non-sensitive row/table counts, removes only that disposable database, and writes a mode-`0600` verification manifest. Before acquiring the shared host lock it validates that `.state` and `backups` are real directories, then uses the constrained deployment-ownership sudo boundary to restore only those directories and an existing regular `mutation.lock` to the current `deploy` UID/GID; it never removes or replaces a lock inode. The workflow retrieves only the encrypted dump and verification metadata, verifies the SHA-256 digest, and uploads them as a 30-day Gitea Actions artifact with compression disabled. The artifact-backed `recovery_reference`, `recovery_created_at`, and `restore_verified=true` output are the inputs for **Migrate Production Database**. Do not use the attestation if backup, restore, retrieval, digest validation, ownership normalization, or artifact upload fails.
|
||||
The host script writes an AES-256-CBC/PBKDF2 encrypted custom-format PostgreSQL dump under `/opt/jyotisha-production/backups`, restores it into a uniquely named disposable database, validates non-sensitive row/table counts, removes only that disposable database, and writes a mode-`0600` verification manifest. Before acquiring the shared host lock it validates that `.state` and `backups` are real directories, then reuses the existing passwordless Docker boundary to run the already-loaded PostgreSQL image with no network, a read-only root filesystem, all capabilities dropped except `CHOWN`, and only those two directories bind-mounted. That helper restores only the directory mount points and an existing regular `mutation.lock` to the current `deploy` UID/GID; it never recursively changes backup files or removes or replaces a lock inode. The workflow retrieves only the encrypted dump and verification metadata, verifies the SHA-256 digest, and uploads them as a 30-day Gitea Actions artifact with compression disabled. The artifact-backed `recovery_reference`, `recovery_created_at`, and `restore_verified=true` output are the inputs for **Migrate Production Database**. Do not use the attestation if backup, restore, retrieval, digest validation, ownership normalization, or artifact upload fails.
|
||||
|
||||
Never restore over `jyotisha`, delete the PostgreSQL volume, print `.env.production.database`, expose `PRODUCTION_BACKUP_ENCRYPTION_KEY`, or substitute a staging recovery artifact. The encrypted local archive is preserved for repair; the Actions artifact supplies the required off-host copy.
|
||||
|
||||
|
||||
@@ -35,14 +35,38 @@ done
|
||||
install -d -m 700 "$state_directory" "$backup_directory"
|
||||
deployment_uid="$(id -u)"
|
||||
deployment_gid="$(id -g)"
|
||||
sudo -n chown "$deployment_uid:$deployment_gid" "$state_directory" "$backup_directory"
|
||||
chmod 700 "$state_directory" "$backup_directory"
|
||||
ownership_targets=("$state_directory" "$backup_directory")
|
||||
if [ -e "$lock_file" ]; then
|
||||
[ -f "$lock_file" ] && [ ! -L "$lock_file" ] || {
|
||||
echo "production mutation lock is unsafe" >&2
|
||||
exit 1
|
||||
}
|
||||
sudo -n chown "$deployment_uid:$deployment_gid" "$lock_file"
|
||||
ownership_targets+=("$lock_file")
|
||||
fi
|
||||
|
||||
mapfile -t postgres_containers < <(
|
||||
sudo -n docker ps -q \
|
||||
--filter 'label=com.docker.compose.project=jyotisha-production' \
|
||||
--filter 'label=com.docker.compose.service=postgres'
|
||||
)
|
||||
[ "${#postgres_containers[@]}" -eq 1 ] || {
|
||||
echo "expected exactly one running production PostgreSQL container" >&2
|
||||
exit 1
|
||||
}
|
||||
postgres_container="${postgres_containers[0]}"
|
||||
ownership_image="$(sudo -n docker inspect --format '{{.Image}}' "$postgres_container")"
|
||||
[[ "$ownership_image" =~ ^sha256:[0-9a-f]{64}$ ]] || {
|
||||
echo "production PostgreSQL image identity is unsafe" >&2
|
||||
exit 1
|
||||
}
|
||||
sudo -n docker run --rm --pull never --network none --read-only --user 0:0 \
|
||||
--cap-drop ALL --cap-add CHOWN --security-opt no-new-privileges \
|
||||
--volume "$state_directory:$state_directory" \
|
||||
--volume "$backup_directory:$backup_directory" \
|
||||
--entrypoint chown "$ownership_image" \
|
||||
"$deployment_uid:$deployment_gid" "${ownership_targets[@]}"
|
||||
chmod 700 "$state_directory" "$backup_directory"
|
||||
if [ -e "$lock_file" ]; then
|
||||
chmod 600 "$lock_file"
|
||||
fi
|
||||
|
||||
@@ -64,17 +88,6 @@ usage_percent="$(df -Pk "$backup_directory" | awk 'NR == 2 {gsub(/%/, "", $5); p
|
||||
exit 1
|
||||
}
|
||||
|
||||
mapfile -t postgres_containers < <(
|
||||
sudo -n docker ps -q \
|
||||
--filter 'label=com.docker.compose.project=jyotisha-production' \
|
||||
--filter 'label=com.docker.compose.service=postgres'
|
||||
)
|
||||
[ "${#postgres_containers[@]}" -eq 1 ] || {
|
||||
echo "expected exactly one running production PostgreSQL container" >&2
|
||||
exit 1
|
||||
}
|
||||
postgres_container="${postgres_containers[0]}"
|
||||
|
||||
created_compact="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
created_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
backup_basename="production-pre-migration-${created_compact}.dump.enc"
|
||||
|
||||
Reference in New Issue
Block a user